Deep Symbolic Regression
Machine Learning Research

From Sequences to Symbols: Transformers Extend AI's Mathematical Capabilities

Given a sequence of numbers, neural networks have proven adept at discovering a mathematical expression that generates it. New work uses transformers to extend that success to a further class of expressions.
Grokking: A dramatic example of generalization far after overfitting on an algorithmic dataset
Machine Learning Research

Learning After Overfitting: Transformers Continue Learning After Overfitting Data

When a model trains too much, it can overfit, or memorize, the training data, which reduces its ability to analyze similar-but-different inputs. But what if training continues? New work found that overfitting isn’t the end of the line.
Stock Market Simulation using cGANs
Machine Learning Research

Stock-Trading Test Bed: AI system simulates stock market performance.

If you buy or sell stocks, it’s handy to test your strategy before you put real money at risk. Researchers devised a fresh approach to simulating market behavior.
Overview of Graph Hyper Network (GHN-2)
Machine Learning Research

Who Needs Training? Graph neural network selects optimal weights for image tasks.

When you’re training a neural network, it takes a lot of computation to optimize its weights using an iterative algorithm like stochastic gradient descent. Wouldn’t it be great to compute the best parameter values in one pass? A new method takes a substantial step in that direction.
Explanation of Multi-Modal Delay Randomization (MMDR)
Machine Learning Research

Coordinating Robot Limbs: Machine learning improves robot dog reaction time.

A dog doesn’t think twice about fetching a tennis ball, but an autonomous robot typically suffers from delays between perception and action. A new machine-learning model helped a quadruped robot coordinate its sensors and actuators.
Schematic of the model architecture showing the generator with spatial latent vectors
Machine Learning Research

Weather Forecast by GAN: GAN improves short-term rainfall predictions.

A new deep learning technique increased the precision of short-term rainfall forecasts. Researchers developed the Deep Generative Model of Radar (DGMR) to predict amounts of precipitation up to two hours in advance.
Illustration of how different data split strategies partition the labelled data
Machine Learning Research

Fine-Tune Your Fine-Tuning: New method optimizes training for few shot NLP models.

Let’s say you have a pretrained language model and a small amount of data to fine-tune it to answer yes-or-no questions. Should you fine-tune it to classify yes/no or to fill in missing words — both viable approaches that are likely to yield different results?
 Service oriented architecture (SOA) implementation of the Ride Allocation app
Machine Learning Research

To Flow or Not to Flow: Building more efficient networked machine learning.

Networked software is often built using a service-oriented architecture, but networked machine learning applications may be easier to manage using a different programming style.
Diagram with info about AlphaCode
Machine Learning Research

Competitive Coder: AI code writing system can compete alongside humans.

Programming is hard. Programming competitions are harder. Yet transformers proved themselves up to the task.
The performance of different downstream (DS)
Machine Learning Research

The Limits of Pretraining: More pretraining doesn't guarantee a better fine-tuned AI.

The higher the accuracy of a pretrained model, the better its performance after fine-tuning, right? Not necessarily. Researchers conducted a meta-analysis of image-recognition experiments and performed some of their own.
Diagram with automated decision systems
Machine Learning Research

Roadblocks to Regulation: Why laws to regulate AI usually fail.

Most U.S. state agencies use AI without limits or oversight. An investigative report probed reasons why efforts to rein them in have made little headway. Since 2018, nearly every proposed bill aimed at studying or controlling how state agencies use automated decision systems.
Fake face diagram - FaceSynthetics
Machine Learning Research

Fake Faces Are Good Training Data: Synthetic data improves face recognition performance.

Collecting and annotating a dataset of facial portraits is a big job. New research shows that synthetic data can work just as well.
Overview of Mobile-Former | Cross attention over the entire featuremap for the first token in Mobile→Former
Machine Learning Research

High Accuracy at Low Power: An energy efficient method for computer vision

Equipment that relies on computer vision while unplugged — mobile phones, drones, satellites, autonomous cars — need power-efficient models. A new architecture set a record for accuracy per computation.
Transformer Architecture
Machine Learning Research

Transformers See in 3D: Using transformers to visualize depth in 2D images.

Visual robots typically perceive the three-dimensional world through sequences of two-dimensional images, but they don’t always know what they’re looking at. For instance, Tesla’s self-driving system has been known to mistake a full moon for a traffic light.
Schematic of 8-bit optimizers via block-wise dynamic quantization
Machine Learning Research

More Learning With Less Memory: Training large language models using less memory.

Researchers discovered a new way to reduce memory requirements when training large machine learning models. Tim Dettmers and colleagues at University of Washington released 8-bit optimizers that store gradient statistics as 8-bit values, while maintaining the same accuracy.

Subscribe to The Batch

Stay updated with weekly AI News and Insights delivered to your inbox