Graph related to Language Model Analysis (LAMA)
Facebook

What Language Models Know

Watson set a high bar for language understanding in 2011, when it famously whipped human competitors in the televised trivia game show Jeopardy! IBM’s special-purpose AI required around $1 billion. Research suggests that today’s best language models can accomplish similar tasks right off the shelf.
2 min read
Illustration of Facebook AI Research method to compress neural networks
Facebook

Honey, I Shrunk the Network!

Deep learning models can be unwieldy and often impractical to run on smaller devices without major modification. Researchers at Facebook AI Research found a way to compress neural networks with minimal sacrifice in accuracy.
2 min read
Bert and Ernie from Sesame Street
Facebook

BERT Is Back

Less than a month after XLNet overtook BERT, the pole position in natural language understanding changed hands again. RoBERTa is an improved BERT pretraining recipe that beats its forbear, becoming the new state-of-the-art language model — for the moment.
2 min read

Subscribe to The Batch

Stay updated with weekly AI News and Insights delivered to your inbox