GPT-2

18 Posts

A living room made out of cups of coffee: the people, the seats, the chimney, the lamp, all gather around a cozy fire.
GPT-2

One Architecture to Do Them All: Transformer: The AI Architecture That Can Do It All

The transformer architecture extended its reach to a variety of new domains.What happened: Originally developed for natural language processing, transformers are becoming the Swiss Army Knife of deep learning.
2 min read
smaller town bigger tree
GPT-2

Trillions of Parameters: Are AI Models With Trillions of Parameters the New Normal?

The trend toward ever-larger models crossed the threshold from immense to ginormous. Google kicked off 2021 with Switch Transformer, the first published work to exceed a trillion parameters, weighing in at 1.6 trillion.
2 min read
A graph shows the cost in dollars of training large natural language processing models.
GPT-2

Who Can Afford to Train AI?: Cost of AI is Too Expensive for Many Small Companies

The cost of training top-performing machine learning models has grown beyond the reach of smaller companies.
2 min read
Animation showing example questions and answers obtained by a pretrained language model
GPT-2

Ask Me in a Different Way: Prompt Engineering Improves Few-Shot Learning Results

Pretrained language models like GPT-3 have shown notable proficiency in few-shot learning. Given a prompt that includes a few example questions and answers (the shots) plus an unanswered question (the task), such models can generate an accurate answer.
2 min read
Frozen Pretrained Transformer (FPT) explained
GPT-2

Transformers: Smarter Than You Think

The transformer architecture has shown an uncanny ability to model not only language but also images and proteins. New research found that it can apply what it learns from the first domain to the others.
2 min read
Animation of SourceAI working
GPT-2

Robocoders

Language models are starting to take on programming work. SourceAI uses GPT-3 to translate plain-English requests into computer code in 40 programming languages. The French startup is one of several companies that use AI to ease coding.
1 min read
Commercial about The Trevor Lifeline
GPT-2

Chatbots Against Depression

A language model is helping crisis-intervention volunteers practice their suicide-prevention skills. The Trevor Project, a nonprofit organization that operates a 24-hour hotline for LGBTQ youth, uses a “crisis contact simulator” to train its staff in how to talk with troubled teenagers.
1 min read
Model predicting ingredients in a recipe and woman cooking
GPT-2

Cake + Cookie = Cakie

AI may help revolutionize the human diet – or dessert, at least.What’s new: Google applied AI engineer Dale Markowitz and developer advocate Sara Robinson trained a model to predict whether a recipe is a
1 min read
Graphs and data related to language models and image processing
GPT-2

Transforming Pixels

Language models like Bert, Ernie, and Elmo have achieved spectacular results based on clever pre-training approaches. New research applies some of those Sesame Street lessons into image processing.
2 min read
Captures from AI's Got Talent
GPT-2

AI’s Got Talent

Music that features a “singing” koala bear took the prize in one of Europe’s highest-profile AI competitions yet. A team of Australian programmers, designers, and musicians won the inaugural AI Song Contest with a koala-tinged track called “Beautiful the World.”
2 min read
Rendering of simulated environment
GPT-2

OpenAI Under Fire

An icon of idealism in AI stands accused of letting its ambition eclipse its principles. Founded in 2015 to develop artificial general intelligence for the good of humankind, OpenAI swapped its ideals for cash.
2 min read
Excerpt from The Squire, an AI written short film
GPT-2

Here Be Dragons

AI is contributing to paintings, music, and now a whimsical fantasy video. The Squire is an amateur romp through a snowy realm of knights in armor and damsels in distress. The script was composed by AI Dungeon 2, an interactive text-adventure game based on the GPT-2 language model.
1 min read
Illustration of a fireplace with "Happy holidays" cards in English, Spanish and French
GPT-2

Language Models Get Literate

Earlier language models powered by Word2Vec and GloVe embeddings yielded confused chatbots, grammar tools with middle-school reading comprehension, and not-half-bad translations. The latest generation is so good, some people consider it dangerous.
2 min read
Sesame Street characters together
GPT-2

Inside AI’s Muppet Empire

As language models show increasing power, a parallel trend has received less notice: The vogue for naming models after characters in the children’s TV show Sesame Street.
1 min read
Illustration of 4 ghosts floating and 1 person dressed as a ghost
GPT-2

Deepfakes Wreak Havoc

Will AI fakery erode public trust in the key social institutions? Generative models will flood media outlets with convincing but false photos, videos, ads, and news stories. The ensuing crisis of authority will lead to widespread distrust in everything from the financial system to democracy itself.
2 min read

Subscribe to The Batch

Stay updated with weekly AI News and Insights delivered to your inbox