Animated image showing the transformer architecture of processing an image

Transformer Speed-Up Sped Up: How to Speed Up Image Transformers

The transformer architecture is notoriously inefficient when processing long sequences — a problem in processing images, which are essentially long sequences of pixels. One way around this is to break up input images and process the pieces
Animation showing Hierarchical Outlier Detection (HOD)

Oddball Recognition: New Method Identifies Outliers in AI Training Data

Models trained using supervised learning struggle to classify inputs that differ substantially from most of their training data. A new method helps them recognize such outliers.
Results of survey about how AI Engineers vs US public feel about ethical issues

AI Engineers Weigh In on AI Ethics: Survey Shows How AI Engineers Feel About Ethical Issues

Machine learning researchers tend to trust international organizations, distrust military forces, and disagree on how much disclosure is necessary when describing new models, a new study found.
Animation showing gMLP, a simple architecture that performed some language and vision tasks as well as transformers

Perceptrons Are All You Need: Google Brain's Multi-Layer Perceptron Rivals Transformers

The paper that introduced the transformer famously declared, “Attention is all you need.” To the contrary, new work shows you may not need transformer-style attention at all.What’s new: Hanxiao Liu and colleagues at Google
A new framework that helps models “unlearn” information selectively and incrementally

Deep Unlearning: AI Researchers Teach Models to Unlearn Data

Privacy advocates want deep learning systems to forget what they’ve learned. What’s new: Researchers are seeking ways to remove the influence of particular training examples, such as an individual’s personal information, from a trained model without affecting its performance, Wired reported.
Apple's CEO Tim Cook discussing privacy with a Privacy sign above him

User Privacy Versus Child Safety: Apple to scan user phones for images of child abuse.

Apple, which has made a point of its commitment to user privacy, announced that it will scan iPhones for evidence of child abuse. The tech giant will include a machine learning model on the device.
Image recognition examples

Smaller Models, Bigger Biases

Compression methods like parameter pruning and quantization can shrink neural networks for use in devices like smartphones with little impact on accuracy — but they also exacerbate a network’s bias.
Animation showing AlphaFold working

Biomedical Treasure Chest

DeepMind opened access to AlphaFold, a model that finds the shapes of proteins, and to its output so far — a potential cornucopia for biomedical research. The research lab, a division of Google’s parent company Alphabet, made AlphaFold freely available.
Frozen Pretrained Transformer (FPT) explained

Transformers: Smarter Than You Think

The transformer architecture has shown an uncanny ability to model not only language but also images and proteins. New research found that it can apply what it learns from the first domain to the others.
Image showing how object detectors work

I Know It When I See It

Object detectors typically detect only items that were labeled in their training data. A new method liberates them to locate and recognize a much wider variety of objects.
Few-shot Learning with a Universal Template (FLUTE)

Pattern for Efficient Learning

Getting high accuracy out of a classifier trained on a small number of examples is tricky. You might train the model on several large-scale datasets prior to few-shot training, but what if the few-shot dataset includes novel classes? A new method performs well even in that case.
A neural network wrote the blueprint for upcoming computer chips

Computers Making Computers

A neural network wrote the blueprint for upcoming computer chips that will accelerate deep learning itself. Google engineers used a reinforcement learning system to arrange the billions of minuscule transistors in an upcoming version of its Tensor Processing Unit (TPU) chips.
Different models in production

ML in Production: Essential Papers

Deploying models for practical use is an industrial concern that generally goes unaddressed in research. As a result, publications on the subject tend to come from the major AI companies.
Craig Wiley speaking

MLOps for All

Craig Wiley has journeyed from the hand-deployed models of yore to the pinnacle of automated AI. Today, as chief product manager of Google Cloud’s AI services, he’s making advanced tools and processes available to anyone with a credit card.
Animation showing a AI's metaphorical transition to using green energy.

Greener Machine Learning: Techniques for Reducing the Carbon Footprint of NLP Models

A new study suggests tactics for machine learning engineers to cut their carbon emissions. Led by David Patterson, researchers at Google and UC Berkeley found that AI developers can shrink a model’s carbon footprint a thousand-fold by streamlining architecture...

Subscribe to The Batch

Stay updated with weekly AI News and Insights delivered to your inbox