Long-Range Arena
Transformer Variants Head to Head: A benchmark for comparing different AI transformers.
The transformer architecture has inspired a plethora of variations. Yet researchers have used a patchwork of metrics to evaluate their performance, making them hard to compare. New work aims to level the playing field.