Meta

18 Posts

Animation showing 3 main types of data augmentation and random cropping of a picture
Meta

Cookbook for Vision Transformers: A Formula for Training Vision Transformers

Vision Transformers (ViTs) are overtaking convolutional neural networks (CNN) in many vision tasks, but procedures for training them are still tailored for CNNs. New research investigated how various training ingredients affect ViT performance.
2 min read
Animated graphs showing how an ensemble of fine-tuned models can provide better performance.
Meta

Ensemble Models Simplified: New Machine Learning Research Simplifies Ensembles

A CLIP model whose weights were the mean of an ensemble of fine-tuned models performed as well as the ensemble and better than its best-performing constituent.
2 min read
Two randomly cropped pictures
Meta

Tradeoffs for Higher Accuracy

Vision models can be improved by training them on several altered versions of the same image and also by encouraging their weights to be close to zero. Recent research showed that both can have adverse effects that may be difficult to detect.
2 min read
House for sale AD
Meta

U.S. Acts Against Algorithmic Bias

Regulators are forcing Meta (formerly Facebook) to display certain advertisements more evenly across its membership. The United States government compelled Meta to revise its ad-placement system to deliver ads for housing to members regardless of their age, gender, or ethnicity.
2 min read
Metaverse illustration with Meta AI product names
Meta

Meta Decentralizes AI Effort

The future of Big AI may lie with product-development teams. Meta reorganized its AI division. Henceforth, AI teams will report to departments that develop key products.
2 min read
Graph Average across 14 NLP Tasks parameters versus Average Accuracy
Meta

GPT-Free

Itching to get your hands on a fully trained large language model? The wait is over. Meta introduced the OPT family of transformer-based language models with nearly unfettered access to source code and trained weights.
2 min read
Deep Symbolic Regression
Meta

From Sequences to Symbols

Given a sequence of numbers, neural networks have proven adept at discovering a mathematical expression that generates it. New work uses transformers to extend that success to a further class of expressions.
2 min read
AI Research SuperCluster (RSC)
Meta

New Supercomputer on the Block

Facebook’s parent company is staking its future on a new compute cluster. Meta unveiled AI Research SuperCluster (RSC), which is designed to accelerate training of large models for applications like computer vision, natural language processing, and speech recognition.
2 min read
Questionnaire for evaluating AI system vendors
Meta

Standards for Hiring Algorithms

Some of the world’s largest corporations will use standardized criteria to evaluate AI systems that influence hiring and other personnel decisions.
2 min read
A living room made out of cups of coffee: the people, the seats, the chimney, the lamp, all gather around a cozy fire.
Meta

One Architecture to Do Them All: Transformer: The AI Architecture That Can Do It All

The transformer architecture extended its reach to a variety of new domains.What happened: Originally developed for natural language processing, transformers are becoming the Swiss Army Knife of deep learning.
2 min read
An illustration shows a cozy cabin where all the furniture is made out of coffee mugs.
Meta

Transformers Take Over: Transformers Applied to Vision, Language, Video, and More

In 2021, transformers were harnessed to discover drugs, recognize speech, and paint pictures — and much more.
2 min read
Multimodal AI Takes Off: Multimodal Models, such as CLIP and Dall-E, Are Taking Over AI
Meta

Multimodal AI Takes Off: Multimodal Models, such as CLIP and Dall-E, Are Taking Over AI

While models like GPT-3 and EfficientNet, which work on text and images respectively, are responsible for some of deep learning’s highest-profile successes, approaches that find relationships between text and images made impressive
1 min read
A conversation between a human and an open-domain chatbot.
Meta

Long-Haul Chatbot: Facebook Chatbot is Able to Carry on Long Conversations

Facebook released a chatbot that summarizes dialog on the fly and uses the summary to generate further repartee.
2 min read
Example comparing a nonaugmented model (left) to a model with internet-augmentation (right)
Meta

This Chatbot Does Its Research: Facebook Chatbot Uses the Internet to Inform its Answers

Chatbots often respond to human input with incorrect or nonsensical answers. Why not enable them to search for helpful information?
1 min read
Animation showing how the Facebook algorithm awards points to a post
Meta

How Facebook Fills the Feed: Leaked Documents Show How Facebook's Algorithm Works

Facebook’s recommendation algorithm is a closely guarded secret. Newly leaked documents shed light on the company’s formula for prioritizing posts in an individual user’s feed.
3 min read

Subscribe to The Batch

Stay updated with weekly AI News and Insights delivered to your inbox