Year in Review

13 Posts

A living room made out of cups of coffee: the people, the seats, the chimney, the lamp, all gather around a cozy fire.
Year in Review

One Architecture to Do Them All: Transformer: The AI Architecture That Can Do It All

The transformer architecture extended its reach to a variety of new domains.What happened: Originally developed for natural language processing, transformers are becoming the Swiss Army Knife of deep learning.
2 min read
A chorus of reindeers singing carols from a Reindeer Holiday Songbook on a snowy night
Year in Review

Voices for the Voiceless: Generative AI Models Are Creating Voices for Hollywood and Video Games

Musicians and filmmakers adopted AI as a standard part of the audio-production toolbox. What happened: Professional media makers embraced neural networks that generate new sounds and modify old ones. Voice actors bristled.
2 min read
An illustration shows a cozy cabin where all the furniture is made out of coffee mugs.
Year in Review

Transformers Take Over: Transformers Applied to Vision, Language, Video, and More

In 2021, transformers were harnessed to discover drugs, recognize speech, and paint pictures — and much more.
2 min read
Multimodal AI Takes Off: Multimodal Models, such as CLIP and Dall-E, Are Taking Over AI
Year in Review

Multimodal AI Takes Off: Multimodal Models, such as CLIP and Dall-E, Are Taking Over AI

While models like GPT-3 and EfficientNet, which work on text and images respectively, are responsible for some of deep learning’s highest-profile successes, approaches that find relationships between text and images made impressive
1 min read
Giant snowman taking over the city while helicopters try to take it down
Year in Review

Governments Lay Down the Law: Governments Around The World Increasingly Regulate AI

Legislators worldwide wrote new laws — some proposed, some enacted — to rein in societal impacts of automation.What happened: Authorities at all levels ratcheted up regulatory pressure as
2 min read
smaller town bigger tree
Year in Review

Trillions of Parameters: Are AI Models With Trillions of Parameters the New Normal?

The trend toward ever-larger models crossed the threshold from immense to ginormous. Google kicked off 2021 with Switch Transformer, the first published work to exceed a trillion parameters, weighing in at 1.6 trillion.
2 min read
Bookstack and wrapping paper
Year in Review

Writer’s Unblock

Neural networks for natural language processing got bigger, more prolific, and more fun to play with. Language models, which already had grown to gargantuan size, continued to swell, yielding chatbots that mimic AI luminaries and have very strange ideas about horses.
1 min read
Doctor examining a snowman holding a broom
Year in Review

The Model Will See You Now

Institutional hurdles to AI for medicine began to fall, setting the stage for widespread clinical use of deep learning in medical devices and treatments. DeepMind’s AlphaFold model determined the three-dimensional shape of a protein in just hours.
1 min read
Group of people having a snowball fight and covering with a giant Facebook like button
Year in Review

Algorithms Against Disinformation

The worldwide pandemic and a contentious U.S. election whipped up a storm of automated disinformation, and some big AI companies reaped the whirlwind.
2 min read
Tree farm dataset
Year in Review

Representing the Underrepresented

Some of deep learning’s bedrock datasets came under scrutiny as researchers combed them for built-in biases. Researchers found that popular datasets impart biases against socially marginalized groups to trained models due to the ways the datasets were compiled, labeled, and used.
2 min read
Dozens of snowmen with different characteristics
Year in Review

This Snowman Does Not Exist

While generative adversarial networks were infiltrating cultural, social, and scientific spheres, they quietly transformed the web into a bottomless well of synthetic images of . . . well, you name it.
1 min read
Two reindeers with masks on a snowy night
Year in Review

Coping With Covid

AI accelerated the search for a coronavirus vaccine, detected Covid-19 cases, and otherwise softened the pandemic’s blow. Machine learning researchers worldwide scrambled to harness the technology against the coronavirus.
2 min read
Illustration of a fireplace with "Happy holidays" cards in English, Spanish and French
Year in Review

Language Models Get Literate

Earlier language models powered by Word2Vec and GloVe embeddings yielded confused chatbots, grammar tools with middle-school reading comprehension, and not-half-bad translations. The latest generation is so good, some people consider it dangerous.
2 min read

Subscribe to The Batch

Stay updated with weekly AI News and Insights delivered to your inbox