Two images showing RETRO Architecture and Gopher (280B) vs State of the Art
Machine Learning Research

Large Language Models Shrink: Gopher and RETRO Prove Lean Language Models Can Push Boundaries

DeepMind released three papers that push the boundaries — and examine the issues — of large language models.
2 min read
Animated illustration shows the model architecture of a graph neural network.
Machine Learning Research

A Deeper Look at Graphs: Graph Neural Networks Work Better With More Layers

New research shows that drastically increasing the number of layers in a graph neural networks improves its performance on large datasets.
2 min read
A conversation between a human and an open-domain chatbot.
Machine Learning Research

Long-Haul Chatbot: Facebook Chatbot is Able to Carry on Long Conversations

Facebook released a chatbot that summarizes dialog on the fly and uses the summary to generate further repartee.
2 min read
Example comparing a nonaugmented model (left) to a model with internet-augmentation (right)
Machine Learning Research

This Chatbot Does Its Research: Facebook Chatbot Uses the Internet to Inform its Answers

Chatbots often respond to human input with incorrect or nonsensical answers. Why not enable them to search for helpful information?
1 min read
Animated chart shows how AI can help robots locate key spatial coordinates.
Machine Learning Research

Finding Useful Points in Space: Keypoint3D Helps Robots Locate Spatial Coordinates

A new machine learning method aims to improve a machine’s ability to determine and locate points of interest.
2 min read
Animation showing how MERLOT is able to match contextualized captions with their corresponding video frames
Machine Learning Research

Richer Video Representations: Pretraining Method Improves AI's Ability to Understand Video

To understand a movie scene, viewers often must remember or infer previous events and extrapolate potential consequences. New work improved a model’s ability to do the same.
2 min read
Animated video showing an image generator imitating brush strokes
Machine Learning Research

Different Strokes for Robot Folks: Transformer-Based Image Generator Imitates Painters

A neural network can make a photo resemble a painting via neural style transfer, but it can also learn to reproduce an image by applying brush strokes. A new method taught a system this painterly skill without any training data.
2 min read
Series of example of accurate and inaccurate matching images to text
Machine Learning Research

Crawl the Web, Absorb the Bias: Language Models Absorb Biases from Web Training Data

The emerging generation of trillion-parameter models needs datasets of billions of examples, but the most readily available source of examples on that scale — the web — is polluted with bias and antisocial expressions. A new study examines the issue.
2 min read
Animated image showing the transformer architecture of processing an image
Machine Learning Research

Transformer Speed-Up Sped Up: How to Speed Up Image Transformers

The transformer architecture is notoriously inefficient when processing long sequences — a problem in processing images, which are essentially long sequences of pixels. One way around this is to break up input images and process the pieces
1 min read
Animation showing Hierarchical Outlier Detection (HOD)
Machine Learning Research

Oddball Recognition: New Method Identifies Outliers in AI Training Data

Models trained using supervised learning struggle to classify inputs that differ substantially from most of their training data. A new method helps them recognize such outliers.
2 min read
Animated charts showing how AI can learn from simple tasks to harder versions of the same task
Machine Learning Research

More Thinking Solves Harder Problems: AI Can Learn From Simple Tasks to Solve Hard Problems

In machine learning, an easy task and a more difficult version of the same task — say, a maze that covers a smaller or larger area — often are learned separately.
2 min read
Animation showing image-to-image style transfer — mapping process
Machine Learning Research

AI With a Sense of Style: Style Transfer Method Produces Consistent Output in Successive Frames

The process known as image-to-image style transfer — mapping, say, the character of a painting’s brushstrokes onto a photo — can render inconsistent results. When they apply the styles of different artists to the same target
3 min read
Animation showing a simulated football team and how it works
Machine Learning Research

Team Players: Football-Playing AI Blends Individual and Group Skills

Playing a team sport involves a fluid blend of individual and group skills. Researchers integrated both types of action into realistic humanoid agents that play football (known as soccer in the U.S.).
2 min read
Animation showing gMLP, a simple architecture that performed some language and vision tasks as well as transformers
Machine Learning Research

Perceptrons Are All You Need: Google Brain's Multi-Layer Perceptron Rivals Transformers

The paper that introduced the transformer famously declared, “Attention is all you need.” To the contrary, new work shows you may not need transformer-style attention at all.What’s new: Hanxiao Liu and colleagues at Google
2 min read
Animation showing example questions and answers obtained by a pretrained language model
Machine Learning Research

Ask Me in a Different Way: Prompt Engineering Improves Few-Shot Learning Results

Pretrained language models like GPT-3 have shown notable proficiency in few-shot learning. Given a prompt that includes a few example questions and answers (the shots) plus an unanswered question (the task), such models can generate an accurate answer.
2 min read

Subscribe to The Batch

Stay updated with weekly AI News and Insights delivered to your inbox