Photograph of Yale Song
GPT-3

Yale Song: Foundation models for vision

Large models pretrained on immense quantities of text have been proven to provide strong foundations for solving specialized language tasks. My biggest hope for AI in 2022 is...
Yoav Shoham
GPT-3

Yoav Shoham: Language models that reason

I believe that natural language processing in 2022 will re-embrace symbolic reasoning, harmonizing it with the statistical operation of modern neural networks. Let me explain what I mean by this.
Alexei Efros
GPT-3

Alexei Efros: Learning from the ground up

Things are really starting to get going in the field of AI. After many years (decades?!) of focusing on algorithms, the AI community is finally ready to accept the central role of data and the high-capacity models that are capable of taking advantage of this data.
An illustration shows a cozy cabin where all the furniture is made out of coffee mugs.
GPT-3

Transformers Take Over: Transformers Applied to Vision, Language, Video, and More

In 2021, transformers were harnessed to discover drugs, recognize speech, and paint pictures — and much more.
Illustration of giant Christmas tree in a town plaza
GPT-3

Trillions of Parameters: Are AI models with trillions of parameters the new normal?

The trend toward ever-larger models crossed the threshold from immense to ginormous. Google kicked off 2021 with Switch Transformer, the first published work to exceed a trillion parameters, weighing in at 1.6 trillion.
Animation showing GPT-3 in full action
GPT-3

GPT-3 for All: GPT-3 NLP Model is Available for Select Azure Users

Microsoft is making GPT-3 available to selected customers through its Azure cloud service.
Animations that shows how the Google Search Algorithm works with Multimodal AI
GPT-3

Search Goes Multimodal: Google Upgrades its Search Algorithm with Multimodal AI

Google will upgrade its search engine with a new model that tracks the relationships between words, images, and, in time, videos — the first fruit of its latest research into multimodal machine learning and multilingual language modeling.
Animation showing example questions and answers obtained by a pretrained language model
GPT-3

Ask Me in a Different Way: Prompt Engineering Improves Few-Shot Learning Results

Pretrained language models like GPT-3 have shown notable proficiency in few-shot learning. Given a prompt that includes a few example questions and answers (the shots) plus an unanswered question (the task), such models can generate an accurate answer.
Series of images showing some of the findings of the new study by researchers at Stanford’s Human AI Institute
GPT-3

Weak Foundations Make Weak Models: Foundation AI Models Pass Flaws to Fine-Tuned Variants

A new study examines a major strain of recent research: huge models pretrained on immense quantities of uncurated, unlabeled data and then fine-tuned on a smaller, curated corpus.
Animation of SourceAI working
GPT-3

Robocoders: How SourceAI uses GPT-3 to write code in 40 languages.

Language models are starting to take on programming work. SourceAI uses GPT-3 to translate plain-English requests into computer code in 40 programming languages. The French startup is one of several companies that use AI to ease coding.
Animation showing a AI's metaphorical transition to using green energy.
GPT-3

Greener Machine Learning: Here's how AI models can shrink their carbon footprints.

A new study suggests tactics for machine learning engineers to cut their carbon emissions. Led by David Patterson, researchers at Google and UC Berkeley found that AI developers can shrink a model’s carbon footprint a thousand-fold by streamlining architecture...
GPT-Neo related animation
GPT-3

Language Models Want to Be Free: How EleutherAI is developing a GPT-3 clone.

A grassroots research collective aims to make a GPT-3 clone that’s available to everyone. EleutherAI, a loose-knit group of independent researchers, is developing GPT-Neo, an open source, free-to-use version of OpenAI’s gargantuan language model.
Ilya Sutskever
GPT-3

Ilya Sutskever: OpenAI’s co-founder on building multimodal AI models

The past year was the first in which general-purpose models became economically useful. GPT-3, in particular, demonstrated that large language models have surprising linguistic competence and the ability to perform a wide variety of useful tasks.
Bookstack and wrapping paper
GPT-3

Writer’s Unblock: Language models keep getting bigger and better.

Neural networks for natural language processing got bigger, more prolific, and more fun to play with. Language models, which already had grown to gargantuan size, continued to swell, yielding chatbots that mimic AI luminaries and have very strange ideas about horses.
Animations depicting benchmarking, datasets and best practices
GPT-3

Prosperity of the Commons: Tools from MLCommons for improved model development

A new consortium of companies, schools, and research labs is building open tools for next-generation machine learning. MLCommons aims to foster innovation in machine learning by developing new benchmarks, datasets, and best practices.

Subscribe to The Batch

Stay updated with weekly AI News and Insights delivered to your inbox