Illustration of a broken heart with a smirk in the middle
Transformer

Outing Hidden Hatred: Facebook uses NLP and Vision AI to Detect Hateful Posts

Facebook uses automated systems to block hate speech, but hateful posts can slip through when seemingly benign words and pictures combine to create a nasty message. The social network is tackling this problem by enhancing AI’s ability to recognize context.
Illustration of two translators on a scale
Transformer

Choosing Words Carefully: NLP for Translation Chooses the Right Synonym

The words “big” and “large” have similar meanings, but they aren’t always interchangeable: You wouldn’t refer to an older, male sibling as your “large brother” (unless you meant to be cheeky). Choosing among words with similar meanings is critical in language tasks like translation.
Talking bubbles inside talking bubbles
Transformer

Bigger is Better

Natural language processing lately has come to resemble an arms race, as the big AI companies build models that encompass ever larger numbers of parameters. Microsoft recently held the record — but not for long.
Generative BST example and graph
Transformer

Big Bot Makes Small Talk

Facebook recently rolled out its entry in the World’s Biggest Chatbot sweepstakes. In keeping with the company’s social-networking dominance, the bot is designed to excel at chitchat on any subject.
A chatbot called Meena and a graph comparing it with other chatbot services
Transformer

Toward Open-Domain Chatbots: Meena Scores High on System for Grading NLP Chatbots

Progress in language models is spawning a new breed of chatbots and, unlike their narrow-domain forebears, they have the gift of gab. Recent research tests the limits of conversational AI.
Series of images related to Jukebox, a deep learning system by OpenAI
Transformer

Roll Over, Beyoncé

A new generative model croons like Elvis and raps like Eminem. It might even make you think you’re listening to a lost demo by the Beatles. OpenAI released Jukebox, a deep learning system that has generated thousands of songs in styles from country to metal and soul.
Simplified depiction of LSH Attention
Transformer

Transformers Transformed

Transformer networks have revolutionized natural language processing, but they hog processor cycles and memory. New research demonstrates a more frugal variation.
Capture of a chatbot telling jokes developed by Google Brain
Transformer

Bot Comic

Androids may not dream of electric sheep, but some crack jokes about horses and cows. Meena, a 2.6-billion parameter chatbot developed by Google Brain, showed impressive conversational ability, discussing a variety of topics.
Math equations represented as trees
Transformer

Neural Networks Study Math

In tasks that involve generating natural language, neural networks often map an input sequence of words to an output sequence of words. Facebook researchers used a similar technique on sequences of mathematical symbols, training a model to map math problems to math solutions.
Excerpt from The Squire, an AI written short film
Transformer

Here Be Dragons

AI is contributing to paintings, music, and now a whimsical fantasy video. The Squire is an amateur romp through a snowy realm of knights in armor and damsels in distress. The script was composed by AI Dungeon 2, an interactive text-adventure game based on the GPT-2 language model.
Single Headed Attention RNN (SHA-RNN)
Transformer

Language Modeling on One GPU

The latest large, pretrained language models rely on trendy layers based on transformer networks. New research shows that these newfangled layers may not be necessary.
Yann LeCun
Transformer

Yann LeCun: Learning From Observation

How is it that many people learn to drive a car fairly safely in 20 hours of practice, while current imitation learning algorithms take hundreds of thousands of hours, and reinforcement learning algorithms take millions of hours? Clearly we’re missing something big.
Illustration of a fireplace with "Happy holidays" cards in English, Spanish and French
Transformer

Natural Language Processing Models Get Literate: Top NLP Advances in 2019

Earlier language models powered by Word2Vec and GloVe embeddings yielded confused chatbots, grammar tools with middle-school reading comprehension, and not-half-bad translations. The latest generation is so good, some people consider it dangerous.
Sesame Street characters together
Transformer

Inside AI’s Muppet Empire: Why Are So Many NLP Models Named After Muppets?

As language models show increasing power, a parallel trend has received less notice: The vogue for naming models after characters in the children’s TV show Sesame Street.
Automatically generated text summary from FactCC with misleading facts highlighted in different colors.
Transformer

Keeping the Facts Straight: NLP System FactCC Fact Checks Texts

Automatically generated text summaries are becoming common in search engines and news websites. But existing summarizers often mix up facts. For instance, a victim’s name might get switched for the perpetrator’s.

Subscribe to The Batch

Stay updated with weekly AI News and Insights delivered to your inbox