A generative adversarial network (GAN)
Transformer

Image Generation Transformed

A recent generative adversarial network (GAN) produced more coherent images using modified transformers that replaced fully connected layers with convolutional layers. A new GAN achieved a similar end using transformers in their original form.
CogView home website
Transformer

Large Language Models for Chinese

Researchers unveiled competition for the reigning large language model GPT-3. Four models collectively called Wu Dao were described by Beijing Academy of Artificial Intelligence, a research collective funded by the Chinese government, according to Synced Review.
Examples of image generators using GANsformer
Transformer

Attention for Image Generation

Attention quantifies how each part of one input affects the various parts of another. Researchers added a step that reverses this comparison to produce more convincing images.
Commercial about The Trevor Lifeline
Transformer

Chatbots Against Depression

A language model is helping crisis-intervention volunteers practice their suicide-prevention skills. The Trevor Project, a nonprofit organization that operates a 24-hour hotline for LGBTQ youth, uses a “crisis contact simulator” to train its staff in how to talk with troubled teenagers.
Graph showing information about different transformer models
Transformer

Transformer Variants Head to Head

The transformer architecture has inspired a plethora of variations. Yet researchers have used a patchwork of metrics to evaluate their performance, making them hard to compare. New work aims to level the playing field.
System Oscar+ working
Transformer

Sharper Eyes For Vision+Language

Models that interpret the interplay of words and images tend to be trained on richer bodies of text than images. Recent research worked toward giving such models a more balanced knowledge of the two domains.
Different graphs showing switch transformer data
Transformer

Bigger, Faster Transformers: Google's Switch Transformer uses MoE for Efficient NLP

Performance in language tasks rises with the size of the model — yet, as a model’s parameter count rises, so does the time it takes to render output. New work pumps up the number of parameters without slowing down the network.
Series of images showing improvements in a multilingual language translator
Transformer

Better Zero-Shot Translations

Train a multilingual language translator to translate between Spanish and English and between English and German, and it may be able to translate directly between Spanish and German as well. New work proposes a simple path to better machine translation between languages.
Graphs and data related to visualized tokens (or vokens)
Transformer

Better Language Through Vision

For children, associating a word with a picture that illustrates it helps them learn the word’s meaning. Research aims to do something similar for machine learning models. Researchers improved a BERT model’s performance on some language tasks by training it on a large dataset of image-word pairs.
GPT-Neo related animation
Transformer

Language Models Want to Be Free: EleutherAI is Making an Open Source Version of GPT-3

A grassroots research collective aims to make a GPT-3 clone that’s available to everyone. EleutherAI, a loose-knit group of independent researchers, is developing GPT-Neo, an open source, free-to-use version of OpenAI’s gargantuan language model.
Data and graphs related to a new model capable of detecting tremors
Transformer

Quake Watch

Detecting earthquakes is an important step toward warning surrounding communities that damaging seismic waves may be headed their way. A new model detects tremors and provides clues to their epicenter.
Data related to adversarial learning
Transformer

Adversarial Helper

Models that learn relationships between images and words are gaining a higher profile. New research shows that adversarial learning, usually a way to make models robust to deliberately misleading inputs, can boost vision-and-language performance.
AI-generated images with the model DALL-E
Transformer

Tell Me a Picture

Two new models show a surprisingly sharp sense of the relationship between words and images. OpenAI, the for-profit research lab, announced a pair of models that have produced impressive results in multimodal learning: DALL·E.
Ilya Sutskever
Transformer

Ilya Sutskever: Fusion of Language and Vision

The past year was the first in which general-purpose models became economically useful. GPT-3, in particular, demonstrated that large language models have surprising linguistic competence and the ability to perform a wide variety of useful tasks.
Matthew Mattina
Transformer

Matthew Mattina: Life- Saving Models in Your Pocket

Look at the tip of a standard #2 pencil. Now, imagine performing over one trillion multiplication operations in the area of that pencil tip every second. This can be accomplished using today’s 7nm semiconductor technology.

Subscribe to The Batch

Stay updated with weekly AI News and Insights delivered to your inbox