Yoav Shoham
Language

Yoav Shoham: Language Models That Reason

I believe that natural language processing in 2022 will re-embrace symbolic reasoning, harmonizing it with the statistical operation of modern neural networks. Let me explain what I mean by this.
2 min read
Abeba Birhane
Language

Abeba Birhane: Clean Up Web Datasets

From language to vision models, deep neural networks are marked by improved performance, higher efficiency, and better generalizations. Yet, these systems are also marked by perpetuation of bias and injustice.
3 min read
A living room made out of cups of coffee: the people, the seats, the chimney, the lamp, all gather around a cozy fire.
Language

One Architecture to Do Them All: Transformer: The AI Architecture That Can Do It All

The transformer architecture extended its reach to a variety of new domains.What happened: Originally developed for natural language processing, transformers are becoming the Swiss Army Knife of deep learning.
2 min read
An illustration shows a cozy cabin where all the furniture is made out of coffee mugs.
Language

Transformers Take Over: Transformers Applied to Vision, Language, Video, and More

In 2021, transformers were harnessed to discover drugs, recognize speech, and paint pictures — and much more.
2 min read
Multimodal AI Takes Off: Multimodal Models, such as CLIP and Dall-E, Are Taking Over AI
Language

Multimodal AI Takes Off: Multimodal Models, such as CLIP and Dall-E, Are Taking Over AI

While models like GPT-3 and EfficientNet, which work on text and images respectively, are responsible for some of deep learning’s highest-profile successes, approaches that find relationships between text and images made impressive
1 min read
smaller town bigger tree
Language

Trillions of Parameters: Are AI Models With Trillions of Parameters the New Normal?

The trend toward ever-larger models crossed the threshold from immense to ginormous. Google kicked off 2021 with Switch Transformer, the first published work to exceed a trillion parameters, weighing in at 1.6 trillion.
2 min read
Two images showing RETRO Architecture and Gopher (280B) vs State of the Art
Language

Large Language Models Shrink: Gopher and RETRO Prove Lean Language Models Can Push Boundaries

DeepMind released three papers that push the boundaries — and examine the issues — of large language models.
2 min read
A conversation between a human and an open-domain chatbot.
Language

Long-Haul Chatbot: Facebook Chatbot is Able to Carry on Long Conversations

Facebook released a chatbot that summarizes dialog on the fly and uses the summary to generate further repartee.
2 min read
Animated image showing different Zillow listings
Language

Price Prediction Turns Perilous: How Covid Broke Zillow's Pricing Algorithm

The real-estate website Zillow bought and sold homes based on prices estimated by an algorithm — until Covid-19 confounded the model’s predictive power. Zillow, whose core business is providing real-estate information for prospective buyers, shut down its house-flipping division after...
2 min read
A graph shows the cost in dollars of training large natural language processing models.
Language

Who Can Afford to Train AI?: Cost of AI is Too Expensive for Many Small Companies

The cost of training top-performing machine learning models has grown beyond the reach of smaller companies.
2 min read
Example comparing a nonaugmented model (left) to a model with internet-augmentation (right)
Language

This Chatbot Does Its Research: Facebook Chatbot Uses the Internet to Inform its Answers

Chatbots often respond to human input with incorrect or nonsensical answers. Why not enable them to search for helpful information?
1 min read
Animation showing GPT-3 in full action
Language

GPT-3 for All: GPT-3 is Available for Select Azure Users

Microsoft is making GPT-3 available to selected customers through its Azure cloud service.
2 min read
First image showing the Google Tensor chip. Second image showing the Google Pixel 6 phone
Language

Competition Heats Up in Mobile AI: Google Designed Its Own Tensor AI Chip for Smartphones

Google designed its own AI chip for its new smartphone — a snub to Qualcomm, the dominant chip vendor in Android phones. What’s new: Google debuted the Tensor chip last week
2 min read
Animation showing how MERLOT is able to match contextualized captions with their corresponding video frames
Language

Richer Video Representations: Pretraining Method Improves AI's Ability to Understand Video

To understand a movie scene, viewers often must remember or infer previous events and extrapolate potential consequences. New work improved a model’s ability to do the same.
2 min read
Illustration showing a witch cooking a copy of the Mona Lisa wearing a witch hat)
Language

Artistry Is Obsolete: Is AI Making Human Artists Obsolete?

Is human creativity being replaced by the synthetic equivalent? The fear: AI is cranking out increasingly sophisticated visual, musical, and literary works. AI-generated media will flood the market, squeezing out human artists and depriving the world of their creativity.
2 min read

Subscribe to The Batch

Stay updated with weekly AI News and Insights delivered to your inbox