Diagram with automated decision systems
transformers

Roadblocks to Regulation

Most U.S. state agencies use AI without limits or oversight. An investigative report probed reasons why efforts to rein them in have made little headway. Since 2018, nearly every proposed bill aimed at studying or controlling how state agencies use automated decision systems.
2 min read
Overview of Mobile-Former | Cross attention over the entire featuremap for the first token in Mobile→Former
transformers

High Accuracy at Low Power

Equipment that relies on computer vision while unplugged — mobile phones, drones, satellites, autonomous cars — need power-efficient models. A new architecture set a record for accuracy per computation.
2 min read
Transformer Architecture
transformers

Transformers See in 3D

Visual robots typically perceive the three-dimensional world through sequences of two-dimensional images, but they don’t always know what they’re looking at. For instance, Tesla’s self-driving system has been known to mistake a full moon for a traffic light.
3 min read
Illustration: Board game pieces and puzzle pieces
transformers

How to Keep Up in a Changing Field

Machine learning changes fast. Take natural language processing. Word2vec, introduced in 2013, quickly replaced one-hot encoding with word embeddings. Transformers revolutionized the field in 2017 by parallelizing the previously sequential training process.
3 min read
A living room made out of cups of coffee: the people, the seats, the chimney, the lamp, all gather around a cozy fire.
transformers

One Architecture to Do Them All: Transformer: The AI Architecture That Can Do It All

The transformer architecture extended its reach to a variety of new domains.What happened: Originally developed for natural language processing, transformers are becoming the Swiss Army Knife of deep learning.
2 min read
An illustration shows a cozy cabin where all the furniture is made out of coffee mugs.
transformers

Transformers Take Over: Transformers Applied to Vision, Language, Video, and More

In 2021, transformers were harnessed to discover drugs, recognize speech, and paint pictures — and much more.
2 min read
Multimodal AI Takes Off: Multimodal Models, such as CLIP and Dall-E, Are Taking Over AI
transformers

Multimodal AI Takes Off: Multimodal Models, such as CLIP and Dall-E, Are Taking Over AI

While models like GPT-3 and EfficientNet, which work on text and images respectively, are responsible for some of deep learning’s highest-profile successes, approaches that find relationships between text and images made impressive
1 min read
smaller town bigger tree
transformers

Trillions of Parameters: Are AI Models With Trillions of Parameters the New Normal?

The trend toward ever-larger models crossed the threshold from immense to ginormous. Google kicked off 2021 with Switch Transformer, the first published work to exceed a trillion parameters, weighing in at 1.6 trillion.
2 min read
Two images showing RETRO Architecture and Gopher (280B) vs State of the Art
transformers

Large Language Models Shrink: Gopher and RETRO Prove Lean Language Models Can Push Boundaries

DeepMind released three papers that push the boundaries — and examine the issues — of large language models.
2 min read
A conversation between a human and an open-domain chatbot.
transformers

Long-Haul Chatbot: Facebook Chatbot is Able to Carry on Long Conversations

Facebook released a chatbot that summarizes dialog on the fly and uses the summary to generate further repartee.
2 min read
Example comparing a nonaugmented model (left) to a model with internet-augmentation (right)
transformers

This Chatbot Does Its Research: Facebook Chatbot Uses the Internet to Inform its Answers

Chatbots often respond to human input with incorrect or nonsensical answers. Why not enable them to search for helpful information?
1 min read
Animation showing GPT-3 in full action
transformers

GPT-3 for All: GPT-3 is Available for Select Azure Users

Microsoft is making GPT-3 available to selected customers through its Azure cloud service.
2 min read
Animation showing how MERLOT is able to match contextualized captions with their corresponding video frames
transformers

Richer Video Representations: Pretraining Method Improves AI's Ability to Understand Video

To understand a movie scene, viewers often must remember or infer previous events and extrapolate potential consequences. New work improved a model’s ability to do the same.
2 min read
Illustration showing a witch cooking a copy of the Mona Lisa wearing a witch hat)
transformers

Artistry Is Obsolete: Is AI Making Human Artists Obsolete?

Is human creativity being replaced by the synthetic equivalent? The fear: AI is cranking out increasingly sophisticated visual, musical, and literary works. AI-generated media will flood the market, squeezing out human artists and depriving the world of their creativity.
2 min read
Halloween family portrait showing the inheritance of some spooky characteristics
transformers

New Models Inherit Old Flaws: AI Models May Inherit Flaws From Previous Systems

Is AI becoming inbred? The fear: The best models increasingly are fine-tuned versions of a small number of so-called foundation models that were pretrained on immense quantities of data scraped from the web.
1 min read

Subscribe to The Batch

Stay updated with weekly AI News and Insights delivered to your inbox