Proof Search Tree
transformers

The Proof Is in the Network

OpenAI’s Generative Pre-Trained Transformer (GPT) architecture has created coherent essays, images, and code. Now it generates mathematical proofs as well.
2 min read
AI medical chatbot having a conversation with a patient
transformers

GPT-3 Is No MD

The world’s most sophisticated language model won’t replace your doctor anytime soon. Researchers at Nabla, an AI-enabled healthcare platform, found that GPT-3 lacks the logical reasoning skills to be a useful medical chatbot.
1 min read
Illustration of two witches with half a pumpkin each and the moon behind them
transformers

The AI Community Splinters

Will international rivalries fragment international cooperation in machine learning? Countries competing for AI dominance will lash out at competitors.
2 min read
Graphs related to different attention mechanisms
transformers

More Efficient Transformers

As transformer networks move to the fore in applications from language to vision, the time it takes them to crunch longer sequences becomes a more pressing issue. A new method lightens the computational load using sparse attention.
2 min read
Graphs with data related to Microsoft's library DeepSpeed
transformers

Toward 1 Trillion Parameters

An open source library could spawn trillion-parameter neural networks and help small-time developers build big-league models. Microsoft upgraded DeepSpeed, a library that accelerates the PyTorch deep learning framework.
2 min read
Bert (muppet) and information related to BERT (transformer-based machine learning technique)
transformers

Do Muppets Have Common Sense?

Two years after it pointed a new direction for language models, Bert still hovers near the top of several natural language processing leaderboards. A new study considers whether Bert simply excels at tracking word order or or learns something closer to common sense.
2 min read
Graphs and data related to transformer networks
transformers

The Transformation Continues

Transformer networks are gaining popularity as a high-accuracy alternative to recurrent neural networks. But they can run slowly when they’re applied to long sequences.
2 min read
Graphs and data related to language models and image processing
transformers

Transforming Pixels

Language models like Bert, Ernie, and Elmo have achieved spectacular results based on clever pre-training approaches. New research applies some of those Sesame Street lessons into image processing.
2 min read
Examples of clothes image-text combo search
transformers

That Online Boutique, But Smarter

Why search for “a cotton dress shirt with button-down collar, breast pockets, barrel cuffs, scooped hem, and tortoise shell buttons in grey” when a photo and the words “that shirt, but grey” will do the trick? A new network understands the image-text combo.
2 min read
Examples and explanation of an automatic headline generation
transformers

AI Makes Headlines

Which headline was written by a computer? A: FIFA to Decide on 2022 World Cup in March B: Decision in March on 48-team 2022 World Cup, Says Infantino
2 min read
Examples of detection of animals in images using Detection Transformer (DETR).
transformers

Computer Vision Transformed

The transformer architecture that has shaken up natural language processing may replace recurrent layers in object detection networks. A Facebook team led by Nicolas Carion and Francisco Massa simplified object detection pipelines by using transformers, yielding Detection Transformer (DETR).
1 min read
Virtual bot speaking
transformers

Bots Don’t Need Social Distancing

A chatbot is providing companionship for the locked-down and lonely. Downloads of Replika, a chatbot designed to be a virtual friend, have spiked during the coronavirus pandemic, reports the New York Times.
1 min read
Illustration of a broken heart with a smirk in the middle
transformers

Outing Hidden Hatred

Facebook uses automated systems to block hate speech, but hateful posts can slip through when seemingly benign words and pictures combine to create a nasty message. The social network is tackling this problem by enhancing AI’s ability to recognize context.
2 min read
Illustration of two translators on a scale
transformers

Choosing Words Carefully

The words “big” and “large” have similar meanings, but they aren’t always interchangeable: You wouldn’t refer to an older, male sibling as your “large brother” (unless you meant to be cheeky). Choosing among words with similar meanings is critical in language tasks like translation.
2 min read
Talking bubbles inside talking bubbles
transformers

Bigger is Better

Natural language processing lately has come to resemble an arms race, as the big AI companies build models that encompass ever larger numbers of parameters. Microsoft recently held the record — but not for long.
2 min read

Subscribe to The Batch

Stay updated with weekly AI News and Insights delivered to your inbox