Frozen Pretrained Transformer (FPT)
Transformers: Smarter Than You Think
The transformer architecture has shown an uncanny ability to model not only language but also images and proteins. New research found that it can apply what it learns from the first domain to the others.