Different graphs showing switch transformer data

Bigger, Faster Transformers: Increasing parameters without slowing down transformers

Performance in language tasks rises with the size of the model — yet, as a model’s parameter count rises, so does the time it takes to render output. New work pumps up the number of parameters without slowing down the network.
Art pieces with subjective commentary regarding their emotional impact

How Art Makes AI Feel: How an AI model feels about art.

An automated art critic spells out the emotional impact of images. Led by Panos Achlioptas, researchers at Ecole Polytechnique, King Abdullah University, and Stanford University trained a deep learning system to generate subjective interpretations of art.
Series of images showing improvements in a multilingual language translator

Better Zero-Shot Translations: A method for improving transformer NLP translation

Train a multilingual language translator to translate between Spanish and English and between English and German, and it may be able to translate directly between Spanish and German as well. New work proposes a simple path to better machine translation between languages.
Graphs and data related to visualized tokens (or vokens)

Better Language Through Vision: Study improved Bert performance using visual tokens.

For children, associating a word with a picture that illustrates it helps them learn the word’s meaning. Research aims to do something similar for machine learning models. Researchers improved a BERT model’s performance on some language tasks by training it on a large dataset of image-word pairs.
GPT-Neo related animation

Language Models Want to Be Free: How EleutherAI is developing a GPT-3 clone.

A grassroots research collective aims to make a GPT-3 clone that’s available to everyone. EleutherAI, a loose-knit group of independent researchers, is developing GPT-Neo, an open source, free-to-use version of OpenAI’s gargantuan language model.
Data related to a language model that predicts mutations that would enable infectious viruses

The Language of Viruses: Researchers trained a neural net to predict viruses in DNA.

A neural network learned to read the genes of viruses as though they were text. That could enable researchers to page ahead for potentially dangerous mutations. Researchers at MIT trained a language model to predict mutations that would enable infectious viruses to become even more virulent.
Data and graphs related to a new model capable of detecting tremors

Quake Watch: AI model detects earthquakes and estimates epicenters.

Detecting earthquakes is an important step toward warning surrounding communities that damaging seismic waves may be headed their way. A new model detects tremors and provides clues to their epicenter.
Facebook service describing a photo on Instagram

Every Picture Tells a Story: Facebook expands automated alternative text.

Facebook expanded a system of vision, language, and speech models designed to open the social network to users who are visually impaired. A Facebook service that describes photos in a synthesized voice now recognizes 1,200 visual concepts — 10 times more than the previous version.
Data related to adversarial learning

Adversarial Helper: Adversarial learning can improve vision and NLP.

Models that learn relationships between images and words are gaining a higher profile. New research shows that adversarial learning, usually a way to make models robust to deliberately misleading inputs, can boost vision-and-language performance.
AI-generated images with the model DALL-E

Tell Me a Picture: OpenAI's two new multimodal AI models, CLIP and DALL·E

Two new models show a surprisingly sharp sense of the relationship between words and images. OpenAI, the for-profit research lab, announced a pair of models that have produced impressive results in multimodal learning: DALL·E.
Animation alternating sad and happy emojis

Online Clues to Mental Illness: NLP model spots signs of mental illness in Facebook posts.

Can social media posts reveal early signs of mental illness? A new machine learning model shows promising results. Researchers developed a model that analyzes messages and images posted by Facebook users for indicators of psychological problems.
Ilya Sutskever

Ilya Sutskever: OpenAI’s co-founder on building multimodal AI models

The past year was the first in which general-purpose models became economically useful. GPT-3, in particular, demonstrated that large language models have surprising linguistic competence and the ability to perform a wide variety of useful tasks.
Harry Shum

Harry Shum: Tsinghua University’s Harry Shum on how AI is changing creativity

In 2021, I envision that the AI community will create more tools to unleash human creativity. AI will help people across the globe to communicate and express emotions and moods in their own unique ways.
Fei-Fei Li

Stanford professor Fei-Fei on how a national research cloud would boost AI

The United States has been a leader in science and technology for decades, and all nations have benefitted from its innovations. But U.S. leadership in AI is not guaranteed.
Bookstack and wrapping paper

Writer’s Unblock: Language models keep getting bigger and better.

Neural networks for natural language processing got bigger, more prolific, and more fun to play with. Language models, which already had grown to gargantuan size, continued to swell, yielding chatbots that mimic AI luminaries and have very strange ideas about horses.

Subscribe to The Batch

Stay updated with weekly AI News and Insights delivered to your inbox