University College London

5 Posts

Diagram explaining Atlas, a retrieval-augmented language model that exhibits strong few-shot performance on knowledge tasks
University College London

Memorize Less; Retrieve More: How small language models can perform specialized tasks.

Large language models are trained only to predict the next word based on previous ones. Yet, given a modest fine-tuning set, they acquire enough information to learn how to perform tasks such as answering questions.
Sequence of famous arcade games' scenes
University College London

Solve RL With This One Weird Trick: How to get better performance from reinforcement learning.

The previous state-of-the-art model for playing vintage Atari games took advantage of a number of advances in reinforcement learning (RL). The new champion is a basic RL architecture plus a trick borrowed from image generation.
Screen captures of online platform Dynabench
University College London

Dynamic Benchmarks

Benchmarks provide a scientific basis for evaluating model performance, but they don’t necessarily map well to human cognitive abilities. Facebook aims to close the gap through a dynamic benchmarking method that keeps humans in the loop.
Schematic of the architecture used in experiments related to systematic reasoning in deep reinforcement learning
University College London

How Neural Networks Generalize

Humans understand the world by abstraction: If you grasp the concept of grabbing a stick, then you’ll also comprehend grabbing a ball. New work explores deep learning agents’ ability to do the same thing — an important aspect of their ability to generalize.
Graph related to Language Model Analysis (LAMA)
University College London

What Language Models Know

Watson set a high bar for language understanding in 2011, when it famously whipped human competitors in the televised trivia game show Jeopardy! IBM’s special-purpose AI required around $1 billion. Research suggests that today’s best language models can accomplish similar tasks right off the shelf.

Subscribe to The Batch

Stay updated with weekly AI News and Insights delivered to your inbox