Switch Transformers

2 Posts

Graph Average across 14 NLP Tasks parameters versus Average Accuracy
Switch Transformers

GPT-Free

Itching to get your hands on a fully trained large language model? The wait is over.What’s new: Meta introduced the OPT family of transformer-based language models with nearly unfettered access to source code and trained weights. The
2 min read
smaller town bigger tree
Switch Transformers

Trillions of Parameters: Are AI Models With Trillions of Parameters the New Normal?

The trend toward ever-larger models crossed the threshold from immense to ginormous. Google kicked off 2021 with Switch Transformer, the first published work to exceed a trillion parameters, weighing in at 1.6 trillion.
2 min read

Subscribe to The Batch

Stay updated with weekly AI News and Insights delivered to your inbox