53 Posts

Translating a Mostly Oral Language: How Meta Trained an NLP Model to Translate Hokkein

Translating a Mostly Oral Language: How Meta Trained an NLP Model to Translate Hokkein

Most speech-to-speech translation systems use text as an intermediate mode. So how do you build an automated translator for a language that has no standard written form? A new approach trained neural networks to translate a primarily oral language.
Dependency between compute budget and number of parameters

Right-Sizing Models for the Dataset: Finding the Best Data-To-Parameter Ratio for NLP Models

The route to improving transformer-based language models like GPT-3 and Gopher, which are trained on immense quantities of text scraped from the web, has been to increase their size. But research shows that, given a processing budget, bigger doesn’t necessarily mean better.
A series of screen captures from the Social Sentinel platform

When Safety Becomes Surveillance: Colleges Track Students Using AI Designed to Monitor Mental Health

United States colleges tracked activists using a natural language processing system intended to monitor their mental health.
Plot demonstrating the relative sizes of parallel and monolingual examples

Massively Multilingual Translation: Machine Learning Model Trained to Translate 1,000 Languages

Recent work showed that models for multilingual machine translation can increase the number of languages they translate by scraping the web for pairs of equivalent sentences in different languages. A new study radically expanded the language repertoire through training on untranslated web text.
Different logos from companies like OpenAI,, Jasper and the dollar sign

Generating Investment: Generative AI Startups Raise Hundreds of Millions in Funding

The generative gold rush is on. Venture capitalists are betting hundreds of millions of dollars on startups that use AI to generate images, text, and more, Wired reported.
Technical components of No Language Left Behind and how they fit together

Massively Multilingual Translation: NLP Model Translates 200 Different Languages

Sentence pairs that have equivalent meanings in different languages — typically used to train machine translation systems — have been available in sufficient quantities for only around 100 languages. New work doubled that number and produced a more capable model.
AI-generated image of Joe Rogan interviewing Steve Jobs

All Synthetic, All the Time: Joe Rogan Meets Steve Jobs in an AI-Generated Podcast

For the debut episode of a new podcast series, synthesized a 19-minute interview between the rock-star podcaster and late Apple CEO.
Example of a video produced from a story-like description

Long-Form Videos from Text Stories: Google's Phenaki Generates Long-Form Video from Text

Only a week ago, researchers unveiled a system that generates a few seconds of video based on a text prompt. New work enables a text-to-video system to produce an entire visual narrative from several sentences of text.
Bloom logo

Large Language Models Unbound: BLOOM is the Largest Open Source NLP Model to Date

A worldwide collaboration produced the biggest open source language model to date. BLOOM is a family of language models built by the BigScience Research Workshop, a collective of over 1,000 researchers from 250 institutions around the globe.
A series of graphs show the carbon emissions associated with training AI models.

Cutting the Carbon Cost of Training: A New Tool Helps NLP Models Lower Their Gas Emissions

You can reduce your model’s carbon emissions by being choosy about when and where you train it.
Everlaw's clustering feature organizing thousands of documents

Order in the Court: Machine Learning Tool from Everlaw Finds Legal Evidence

Machine learning is helping lawyers sift through mountains of documents to find evidence. The legal technology company Everlaw launched a clustering feature that automatically organizes up to 25 million documents for lawyers gathering evidence to be used during a trial.
Example of text generated by LaMDA

LaMDA Comes Alive?: Google Engineer Says LaMDA AI is Sentient

A chatbot persuaded at least one person that it has feelings. A senior engineer at Google announced his belief that the company’s latest conversational language model is sentient.
Andrew Ng staring at neural networks

The Batch: Special Issue! Foundational Algorithms, Where They Came From, Where They're Going

Years ago, I had to choose between a neural network and a decision tree learning algorithm. It was necessary to pick an efficient one, because we planned to apply the algorithm to a very large set of users on a limited compute budget.
Man sitting on a tree with a monkey and a gorilla

Decision Trees: From Root to Leaves — Decision Trees for Machine Learning Explained

What kind of beast was Aristotle? The philosopher's follower Porphyry, who lived in Syria during the third century, came up with a logical way to answer the question...
Illustration of a robot with a captain costume

Neural Networks: Find the Function — A Basic Introduction to Neural Networks

Let’s get this out of the way: A brain is not a cluster of graphics processing units, and if it were, it would run software far more complex than the typical artificial neural network. Yet neural networks were inspired by the brain’s architecture.

Subscribe to The Batch

Stay updated with weekly AI News and Insights delivered to your inbox