GPT-2 text generator
Language

Putting Text Generators on a Leash

Despite dramatic recent progress, natural language generation remains an iffy proposition. Even users of the muscular GPT-2 text generator have to press the button a number of times to get sensible output. But researchers are figuring out how to exert greater control over generated text.
 Proportion of examples covered by number of annotators (sorted by number of annotations)
Language

AI Knows Who Labeled the Data

The latest language models are great at answering questions about a given text passage. However, these models are also powerful enough to recognize an individual writer’s style, which can clue them in to the right answers. New research measures such annotator bias in several data sets.
Google text-to-speech logo
Language

The Long and Short of It

Not long ago, text-to-speech systems could read only a sentence at a time, and they were ranked according to their ability to accomplish that limited task. Now that they can orate entire books, we need new benchmarks.
Average Relative WER improvement as a function of the amount of training data
Language

Speech Recognition With an Accent

Models that achieve state-of-the-art performance in automatic speech recognition (ASR) often perform poorly on nonstandard speech. New research offers methods to make ASR more useful to users with heavy accents or speech impairment.
Two people shaking hands
Language

My Chatbot Will Call Your Chatbot

Companies with large numbers of contractual relationships may leave millions of dollars on the table because it’s not practical to customize each agreement. A new startup offers a chatbot designed to claw back that money.
Graph related to Language Model Analysis (LAMA)
Language

What Language Models Know

Watson set a high bar for language understanding in 2011, when it famously whipped human competitors in the televised trivia game show Jeopardy! IBM’s special-purpose AI required around $1 billion. Research suggests that today’s best language models can accomplish similar tasks right off the shelf.
Question from an exam
Language

Smart Students, Dumb Algorithms: NLP Systems Struggle at Grading Essays

A growing number of companies that sell standardized tests are using natural language processing to assess writing skills. Critics contend that these language models don’t make the grade.
OpenAI's GPT-2 results
Language

How to Share Dangerous AI

OpenAI raised eyebrows in February when it announced — and withheld — the full version of its groundbreaking language model, GPT-2. Six months later, the company has re-examined the decision.
Bert and Ernie from Sesame Street
Language

BERT Is Back

Less than a month after XLNet overtook BERT, the pole position in natural language understanding changed hands again. RoBERTa is an improved BERT pretraining recipe that beats its forbear, becoming the new state-of-the-art language model — for the moment.

Subscribe to The Batch

Stay updated with weekly AI News and Insights delivered to your inbox