System Oscar+ working
transformers

Sharper Eyes For Vision+Language

Models that interpret the interplay of words and images tend to be trained on richer bodies of text than images. Recent research worked toward giving such models a more balanced knowledge of the two domains.
2 min read
Different graphs showing switch transformer data
transformers

Bigger, Faster Transformers

Performance in language tasks rises with the size of the model — yet, as a model’s parameter count rises, so does the time it takes to render output. New work pumps up the number of parameters without slowing down the network.
2 min read
Series of images showing improvements in a multilingual language translator
transformers

Better Zero-Shot Translations

Train a multilingual language translator to translate between Spanish and English and between English and German, and it may be able to translate directly between Spanish and German as well. New work proposes a simple path to better machine translation between languages.
2 min read
Graphs and data related to visualized tokens (or vokens)
transformers

Better Language Through Vision

For children, associating a word with a picture that illustrates it helps them learn the word’s meaning. Research aims to do something similar for machine learning models. Researchers improved a BERT model’s performance on some language tasks by training it on a large dataset of image-word pairs.
2 min read
GPT-Neo related animation
transformers

Language Models Want to Be Free

A grassroots research collective aims to make a GPT-3 clone that’s available to everyone. EleutherAI, a loose-knit group of independent researchers, is developing GPT-Neo, an open source, free-to-use version of OpenAI’s gargantuan language model.
1 min read
Data and graphs related to a new model capable of detecting tremors
transformers

Quake Watch

Detecting earthquakes is an important step toward warning surrounding communities that damaging seismic waves may be headed their way. A new model detects tremors and provides clues to their epicenter.
2 min read
Data related to adversarial learning
transformers

Adversarial Helper

Models that learn relationships between images and words are gaining a higher profile. New research shows that adversarial learning, usually a way to make models robust to deliberately misleading inputs, can boost vision-and-language performance.
2 min read
AI-generated images with the model DALL-E
transformers

Tell Me a Picture

Two new models show a surprisingly sharp sense of the relationship between words and images. OpenAI, the for-profit research lab, announced a pair of models that have produced impressive results in multimodal learning: DALL·E.
2 min read
Ilya Sutskever
transformers

Ilya Sutskever: Fusion of Language and Vision

The past year was the first in which general-purpose models became economically useful. GPT-3, in particular, demonstrated that large language models have surprising linguistic competence and the ability to perform a wide variety of useful tasks.
2 min read
Matthew Mattina
transformers

Matthew Mattina: Life- Saving Models in Your Pocket

Look at the tip of a standard #2 pencil. Now, imagine performing over one trillion multiplication operations in the area of that pencil tip every second. This can be accomplished using today’s 7nm semiconductor technology.
2 min read
Data showing how new pretrained language models might learn facts like weight and cost
transformers

The Measure of a Muppet

The latest pretrained language models have shown a remarkable ability to learn facts. A new study drills down on issues of scale, showing that such models might learn the approximate weight of a dog or cost of an apple, at least to the right order of magnitude.
2 min read
Examples of contrastive learning
transformers

Learning From Words and Pictures

It’s expensive to pay doctors to label medical images, and the relative scarcity of high-quality training examples can make it hard for neural networks to learn features that make for accurate diagnoses.
2 min read
Data related to Nvidia's Pay Attention When Required (Par) approach
transformers

Selective Attention

Large transformer networks work wonders with natural language, but they require enormous amounts of computation. New research slashes processor cycles without compromising performance.
1 min read
Proof Search Tree
transformers

The Proof Is in the Network

OpenAI’s Generative Pre-Trained Transformer (GPT) architecture has created coherent essays, images, and code. Now it generates mathematical proofs as well.
2 min read
AI medical chatbot having a conversation with a patient
transformers

GPT-3 Is No MD

The world’s most sophisticated language model won’t replace your doctor anytime soon. Researchers at Nabla, an AI-enabled healthcare platform, found that GPT-3 lacks the logical reasoning skills to be a useful medical chatbot.
1 min read

Subscribe to The Batch

Stay updated with weekly AI News and Insights delivered to your inbox