Gated Transformer-XL (GTrXL)
Melding Transformers with RL: GTrXL combines transformers and reinforcement learning.
Large NLP models like BERT can answer questions about a document thanks to the transformer network, a sequence-processing architecture that retains information across much longer sequences than previous methods. But transformers have had little success in reinforcement learning — until now.