Efficiency

55 Posts

Dependency between compute budget and number of parameters
Efficiency

Right-Sizing Models for the Dataset: Finding the Best Data-To-Parameter Ratio for NLP Models

The route to improving transformer-based language models like GPT-3 and Gopher, which are trained on immense quantities of text scraped from the web, has been to increase their size. But research shows that, given a processing budget, bigger doesn’t necessarily mean better.
App that monitors machinery
Efficiency

If It Ain’t Broke, Fix It: Factories Use AI for Predictive Maintenance

Factories are using AI to warn them when equipment is reaching the breaking point. Services that monitor machinery to predict imminent failure and provide guidance on necessary upkeep are booming, The Wall Street Journal reported.
Illustration shows different self-attention mechanisms used by Transformer-based AI models.
Efficiency

Attention to Rows and Columns: Altering Transformers' Self-Attention Mechanism for Greater Efficiency

A new approach alters transformers' self-attention mechanism to balance computational efficiency with performance on vision tasks.
Animated graphs showing how an ensemble of fine-tuned models can provide better performance.
Efficiency

Ensemble Models Simplified: New Machine Learning Research Simplifies Ensembles

A CLIP model whose weights were the mean of an ensemble of fine-tuned models performed as well as the ensemble and better than its best-performing constituent.
A series of graphs show the carbon emissions associated with training AI models.
Efficiency

Cutting the Carbon Cost of Training: A New Tool Helps NLP Models Lower Their Gas Emissions

You can reduce your model’s carbon emissions by being choosy about when and where you train it.
Two randomly cropped pictures
Efficiency

Tradeoffs for Higher Accuracy: Data Augmentation Plus Weight Decay can Boost Some AI Models

Vision models can be improved by training them on several altered versions of the same image and also by encouraging their weights to be close to zero. Recent research showed that both can have adverse effects that may be difficult to detect.
Masked Auto-Encoder (MAE) explanation
Efficiency

Who Was That Masked Input? Pretraining Method Improves Computer Vision Performance

Researchers have shown that it’s possible to train a computer vision model effectively on around 66 percent of the pixels in each training image. New work used 25 percent, saving computation and boosting performance to boot.
Overview of Mobile-Former | Cross attention over the entire featuremap for the first token in Mobile→Former
Efficiency

High Accuracy at Low Power: An energy efficient method for computer vision.

Equipment that relies on computer vision while unplugged — mobile phones, drones, satellites, autonomous cars — need power-efficient models. A new architecture set a record for accuracy per computation.
Multimodal deep learning model
Efficiency

AI Versus the Garbage Heap: How Amazon uses AI to cut waste.

Amazon reported long-term success using machine learning to shrink its environmental footprint. The online retailer developed a system that fuses product descriptions, images, and structured data to decide how an item should be packed for shipping.
Schematic of 8-bit optimizers via block-wise dynamic quantization
Efficiency

More Learning With Less Memory: Training large language models using less memory.

Researchers discovered a new way to reduce memory requirements when training large machine learning models. Tim Dettmers and colleagues at University of Washington released 8-bit optimizers that store gradient statistics as 8-bit values, while maintaining the same accuracy.
Animated charts show the results of a machine learning model that predicts the best depression drugs for a patient.
Efficiency

Which Drug Helps Your Depression?: AI System Matches Patients With the Right Depression Drug

Deep learning can predict how patients will respond to two antidepressant medicines.
Animated video shows the Flippy 2 autonomous restaurant robot making french fries
Efficiency

Deep Learning for Deep Frying: White Castle Uses Robots to Cook French Fries

Flippy 2, a robotic fry station from California-based Miso Robotics, has been newly deployed in a Chicago White Castle location.
Customs and immigration kiosks for entering South Korea.
Efficiency

When Officials Share Personal Data: South Korea Supplied Foreigner Data to Face Recognition Developers

The government of South Korea is supplying personal data to developers of face recognition algorithms.
A graph shows the cost in dollars of training large natural language processing models.
Efficiency

Who Can Afford to Train AI?: Cost of AI is Too Expensive for Many Small Companies

The cost of training top-performing machine learning models has grown beyond the reach of smaller companies.
An animation of a protein modeled by AlphaFold 2.
Efficiency

DeepMind Doubles Down on AlphaFold: DeepMind Launches Company to Commercialize AlphaFold 2

Isomorphic aims to build its business on AlphaFold 2, an ensemble of neural networks that finds the shapes of protein molecules.

Subscribe to The Batch

Stay updated with weekly AI News and Insights delivered to your inbox