Reinforcement learning is powering a new generation of video game cheaters.
What’s new: Players of Rocket League, a video game that ranks among the world’s most popular esports, are getting trounced by cheaters who use AI models originally developed to train contestants, PC Gamer reported.
The game: Rocket League’s rules are similar to football (known as soccer in the United States): Players aim to force a ball into their opponent’s goal at the other end of an arena — except, rather than kicking the ball, they push it with a race car. Doing so, however, requires mastering the game’s idiosyncratic physics. Players can drive up the arena’s walls, turbo-boost across the pitch, and launch their car into the air.
How it works: The cheat takes advantage of a bot known as Nexto. Developed by AI-savvy players as a training tool, Nexto and similar bots typically include hard-coded restrictions against being used in competitive online play. However, someone customized the bot, enabling it to circumvent the restriction, one of Nexto’s developers revealed in a discussion on Reddit.
- Nexto was trained using RLGym, an API that allows bot-makers to treat Rocket League as a simulation environment for reinforcement learning.
- Its reward function examined physics parameters within the game such as the velocity of the user’s car, its distance to the ball, and where it touches the ball during a pass or shot.
- Nexto learned by playing against itself in approximately 250,000 hours (roughly 29 years 24/7) worth of gameplay, typically playing many accelerated games simultaneously. The developers estimate that its performance matches that of the top 1 percent of players.
- Nexto’s developers are working on a new bot that can learn from gameplay against human players. They plan not to distribute it beyond their core group to prevent cheaters from exploiting it.
- Rocket League developer Psyonix has banned players it determined cheated with bots including Nexto.
Behind the news: Despite reinforcement learning’s ability to master classic games like go and video games like StarCraft II, news of AI-powered cheats has been scant. The developers of Userviz, a cheatbot for first-person shooters that automatically aimed and fired on enemies detected by a YOLO implementation, deleted access to the app after receiving legal notice from video game publisher Activision.
Why it matters: Video games are big business. Rampant cheating could impact a game’s sales by ruining the experience for casual players. Cheating can also tarnish the reputation of games that, like Rocket League, are played professionally, where top players stand to win millions of dollars.
We’re thinking: While we condemn cheating, we applaud anyone who is so motivated to improve their gaming skill that they develop reinforcement learning models to compete against!