Examples of images being produced from noise
Efficiency

Images From Noise: An upgrade for score-based generative AI models.

Generative adversarial networks and souped-up language models aren’t the only image generators around. Researchers recently upgraded an alternative known as score-based generative models.
Graphs comparing SGD + Momentum, Adam and AdaBelief
Efficiency

Striding Toward the Minimum: A faster way to optimize the loss function for deep learning.

When you’re training a deep learning model, it can take days for an optimization algorithm to minimize the loss function. A new approach could save time.
Graphs related to world models
Efficiency

It’s a Small World Model After All: More efficient world models for reinforcement learning

World models, which learn a compressed representation of a dynamic environment like, say, a video game, have delivered top results in reinforcement learning. A new method makes them much smaller.
AI-driven balloon reaching high altitude
Efficiency

How to Drive a Balloon: How high-altitude balloons navigate using AI.

Helium balloons that beam internet service to hard-to-serve areas are using AI to navigate amid high-altitude winds. Loon, the Alphabet division that provides wireless internet via polyethylene blimps.
Illustration of two witches with half a pumpkin each and the moon behind them
Efficiency

The AI Community Splinters: Could geopolitics drive a wedge in the AI community?

Will international rivalries fragment international cooperation in machine learning? Countries competing for AI dominance will lash out at competitors.
Illustration of a neighborhood haunted by an evil pumpkin and a black cat
Efficiency

Giant Models Bankrupt Research: Will training AI become too expensive for most companies?

What if AI requires so much computation that it becomes unaffordable?The fear: Training ever more capable models will become too pricey for all but the richest corporations and government agencies. Rising costs will
Information and components of a battery
Efficiency

Getting a Charge From AI: How battery developers are using AI

Machine learning is helping to design energy cells that charge faster and last longer. Battery developers are using ML algorithms to devise new chemicals, components, and charging techniques faster than traditional techniques allow.
Graphs related to different attention mechanisms
Efficiency

More Efficient Transformers: BigBird is an efficient attention mechanism for transformers.

As transformer networks move to the fore in applications from language to vision, the time it takes them to crunch longer sequences becomes a more pressing issue. A new method lightens the computational load using sparse attention.
Graphs with data related to Microsoft's library DeepSpeed
Efficiency

Toward 1 Trillion Parameters: Microsoft upgrades its DeepSpeed optimization library.

An open source library could spawn trillion-parameter neural networks and help small-time developers build big-league models. Microsoft upgraded DeepSpeed, a library that accelerates the PyTorch deep learning framework.
Data and information related to dropout
Efficiency

Dropout With a Difference: Reduce neural net overfitting without impacting accuracy

The technique known as dropout discourages neural networks from overfitting by deterring them from reliance on particular features. A new approach reorganizes the process to run efficiently on the chips that typically run neural network calculations.
Graphs and data related to transformer networks
Efficiency

The Transformation Continues: Technique boosts transformer performance on long sequences.

Transformer networks are gaining popularity as a high-accuracy alternative to recurrent neural networks. But they can run slowly when they’re applied to long sequences.
Data related to experience replay
Efficiency

Experience Counts: Research proposes an upgrade to experience replay.

If the world changes every second and you take a picture every 10 seconds, you won’t have enough pictures to observe the changes clearly, and storing a series of pictures won’t help. On the other hand, if you take a picture every tenth of a second, then storing a history will help model the world.
Graphs and data related to semi-supervised learning
Efficiency

All Examples Are Not Equal: An algorithm for improved semi-supervised learning

Semi-supervised learning — a set of training techniques that use a small number of labeled examples and a large number of unlabeled examples — typically treats all unlabeled examples the same way. But some examples are more useful for learning than others.
Excerpt from study about models that learn to predict task-specific distance metrics
Efficiency

Misleading Metrics: Advances in metric learning may be illusions.

A growing body of literature shows that some steps in AI’s forward march may actually move sideways. A new study questions advances in metric learning.
Information related to the Once-for-All (OFA) method
Efficiency

Build Once, Run Anywhere: The Once-For-All technique adapts AI models to edge devices.

From server to smartphone, devices with less processing speed and memory require smaller networks. Instead of building and training separate models to run on a variety of hardware, a new approach trains a single network that can be adapted to any device.

Subscribe to The Batch

Stay updated with weekly AI News and Insights delivered to your inbox