Series of images showing some of the findings of the new study by researchers at Stanford’s Human AI Institute
GPT-3

Weak Foundations Make Weak Models: Foundation AI Models Pass Flaws to Fine-Tuned Variants

A new study examines a major strain of recent research: huge models pretrained on immense quantities of uncurated, unlabeled data and then fine-tuned on a smaller, curated corpus.
Animation of SourceAI working
GPT-3

Robocoders

Language models are starting to take on programming work. SourceAI uses GPT-3 to translate plain-English requests into computer code in 40 programming languages. The French startup is one of several companies that use AI to ease coding.
Animation showing a AI's metaphorical transition to using green energy.
GPT-3

Greener Machine Learning: Techniques for Reducing the Carbon Footprint of NLP Models

A new study suggests tactics for machine learning engineers to cut their carbon emissions. Led by David Patterson, researchers at Google and UC Berkeley found that AI developers can shrink a model’s carbon footprint a thousand-fold by streamlining architecture...
GPT-Neo related animation
GPT-3

Language Models Want to Be Free: EleutherAI is Making an Open Source Version of GPT-3

A grassroots research collective aims to make a GPT-3 clone that’s available to everyone. EleutherAI, a loose-knit group of independent researchers, is developing GPT-Neo, an open source, free-to-use version of OpenAI’s gargantuan language model.
Ilya Sutskever
GPT-3

Ilya Sutskever: Fusion of Language and Vision

The past year was the first in which general-purpose models became economically useful. GPT-3, in particular, demonstrated that large language models have surprising linguistic competence and the ability to perform a wide variety of useful tasks.
Bookstack and wrapping paper
GPT-3

Writer’s Unblock

Neural networks for natural language processing got bigger, more prolific, and more fun to play with. Language models, which already had grown to gargantuan size, continued to swell, yielding chatbots that mimic AI luminaries and have very strange ideas about horses.
Animations depicting benchmarking, datasets and best practices
GPT-3

Prosperity of the Commons

A new consortium of companies, schools, and research labs is building open tools for next-generation machine learning. MLCommons aims to foster innovation in machine learning by developing new benchmarks, datasets, and best practices.
AI medical chatbot having a conversation with a patient
GPT-3

GPT-3 Is No MD

The world’s most sophisticated language model won’t replace your doctor anytime soon. Researchers at Nabla, an AI-enabled healthcare platform, found that GPT-3 lacks the logical reasoning skills to be a useful medical chatbot.
Illustration of two witches with half a pumpkin each and the moon behind them
GPT-3

The AI Community Splinters

Will international rivalries fragment international cooperation in machine learning? Countries competing for AI dominance will lash out at competitors.
Information related to OpenAI's model GPT-3
GPT-3

Generation Text: GPT-3 Beta Testers Create Blog Posts, Computer Code, and More

People granted early access to OpenAI’s latest language model are raving about its way with words — and more. Beta testers of GPT-3 are showing off the model’s ability to write business memos, craft blogs, pen tweets, and even generate computer code.
Virtual bot speaking
GPT-3

Bots Don’t Need Social Distancing

A chatbot is providing companionship for the locked-down and lonely. Downloads of Replika, a chatbot designed to be a virtual friend, have spiked during the coronavirus pandemic, reports the New York Times.
Talking bubbles inside talking bubbles
GPT-3

Bigger is Better

Natural language processing lately has come to resemble an arms race, as the big AI companies build models that encompass ever larger numbers of parameters. Microsoft recently held the record — but not for long.

Subscribe to The Batch

Stay updated with weekly AI News and Insights delivered to your inbox