4 Posts

A living room made out of cups of coffee: the people, the seats, the chimney, the lamp, all gather around a cozy fire.

One Architecture to Do Them All: Transformer: The AI architecture that can do it all.

The transformer architecture extended its reach to a variety of new domains.What happened: Originally developed for natural language processing, transformers are becoming the Swiss Army Knife of deep learning.
Graphs, images and data related to the activation function known as ReLU

Upgrade for ReLU: The sin(x) activation function is an alternative to ReLU.

The activation function known as ReLU builds complex nonlinear functions across layers of a neural network, making functions that outline flat faces and sharp edges. But how much of the world breaks down into perfect polyhedra?
Proof Search Tree

The Proof Is in the Network: A transformer model that generates mathematical proofs

OpenAI’s Generative Pre-Trained Transformer (GPT) architecture has created coherent essays, images, and code. Now it generates mathematical proofs as well.
Math equations represented as trees

Neural Networks Study Math: A sequence to sequence model for solving math problems.

In tasks that involve generating natural language, neural networks often map an input sequence of words to an output sequence of words. Facebook researchers used a similar technique on sequences of mathematical symbols, training a model to map math problems to math solutions.

Subscribe to The Batch

Stay updated with weekly AI News and Insights delivered to your inbox