Diagram with automated decision systems
Transformer

Roadblocks to Regulation: Why laws to regulate AI usually fail.

Most U.S. state agencies use AI without limits or oversight. An investigative report probed reasons why efforts to rein them in have made little headway. Since 2018, nearly every proposed bill aimed at studying or controlling how state agencies use automated decision systems.
Overview of Mobile-Former | Cross attention over the entire featuremap for the first token in Mobile→Former
Transformer

High Accuracy at Low Power: An energy efficient method for computer vision.

Equipment that relies on computer vision while unplugged — mobile phones, drones, satellites, autonomous cars — need power-efficient models. A new architecture set a record for accuracy per computation.
Transformer Architecture
Transformer

Transformers See in 3D: Using transformers to visualize depth in 2D images.

Visual robots typically perceive the three-dimensional world through sequences of two-dimensional images, but they don’t always know what they’re looking at. For instance, Tesla’s self-driving system has been known to mistake a full moon for a traffic light.
Illustration: Board game pieces and puzzle pieces
Transformer

How to Keep Up in a Changing Field: How to keep up with a fast-changing industry.

Machine learning changes fast. Take natural language processing. Word2vec, introduced in 2013, quickly replaced one-hot encoding with word embeddings. Transformers revolutionized the field in 2017 by parallelizing the previously sequential training process.
A living room made out of cups of coffee: the people, the seats, the chimney, the lamp, all gather around a cozy fire.
Transformer

One Architecture to Do Them All: Transformer: The AI architecture that can do it all.

The transformer architecture extended its reach to a variety of new domains.What happened: Originally developed for natural language processing, transformers are becoming the Swiss Army Knife of deep learning.
An illustration shows a cozy cabin where all the furniture is made out of coffee mugs.
Transformer

Transformers Take Over: Transformers Applied to Vision, Language, Video, and More

In 2021, transformers were harnessed to discover drugs, recognize speech, and paint pictures — and much more.
Illustration of a woman riding a sled
Transformer

Multimodal AI Takes Off: Multimodal Models, such as CLIP and DALL-E, are taking over AI.

While models like GPT-3 and EfficientNet, which work on text and images respectively, are responsible for some of deep learning’s highest-profile successes, approaches that find relationships between text and images made impressive
Illustration of giant Christmas tree in a town plaza
Transformer

Trillions of Parameters: Are AI models with trillions of parameters the new normal?

The trend toward ever-larger models crossed the threshold from immense to ginormous. Google kicked off 2021 with Switch Transformer, the first published work to exceed a trillion parameters, weighing in at 1.6 trillion.
Two images showing RETRO Architecture and Gopher (280B) vs State of the Art
Transformer

Large Language Models Shrink: Gopher and RETRO prove lean language models can push boundaries.

DeepMind released three papers that push the boundaries — and examine the issues — of large language models.
Google's Decision Transformer
Transformer

Reinforcement Learning Transformed: Transformers succeed at reinforcemend learning tasks.

Transformers have matched or exceeded earlier architectures in language modeling and image classification. New work shows they can achieve state-of-the-art results in some reinforcement learning tasks as well.
A conversation between a human and an open-domain chatbot.
Transformer

Long-Haul Chatbot: Facebook Chatbot is Able to Carry on Long Conversations

Facebook released a chatbot that summarizes dialog on the fly and uses the summary to generate further repartee.
Example comparing a nonaugmented model (left) to a model with internet-augmentation (right)
Transformer

This Chatbot Does Its Research: Facebook Chatbot Uses the Internet to Inform its Answers

Chatbots often respond to human input with incorrect or nonsensical answers. Why not enable them to search for helpful information?
Animation showing GPT-3 in full action
Transformer

GPT-3 for All: GPT-3 NLP Model is Available for Select Azure Users

Microsoft is making GPT-3 available to selected customers through its Azure cloud service.
Animation showing how MERLOT is able to match contextualized captions with their corresponding video frames
Transformer

Richer Video Representations: Pretraining Method Improves AI's Ability to Understand Video

To understand a movie scene, viewers often must remember or infer previous events and extrapolate potential consequences. New work improved a model’s ability to do the same.
Illustration showing a witch cooking a copy of the Mona Lisa wearing a witch hat)
Transformer

Artistry Is Obsolete: Is AI Making Human Artists Obsolete?

Is human creativity being replaced by the synthetic equivalent? The fear: AI is cranking out increasingly sophisticated visual, musical, and literary works. AI-generated media will flood the market, squeezing out human artists and depriving the world of their creativity.

Subscribe to The Batch

Stay updated with weekly AI News and Insights delivered to your inbox