Sony's new AI driver achieves 'reliably superhuman' race times in Gran Turismo

AI agents have outperformed humans in many games, from chess to Go to poker. Now, the machines can claim a new high score on the classic racing video game series Gran Turismo.

Sony today announced that its researchers have developed an AI driver called the GT Sophie that is "reliably superhuman" – capable of beating the top human drivers in Gran Turismo Sport in back-to-back laps. You might think that this is an easy challenge. After all, isn't racing just a matter of speed and reaction time and therefore an easy machine to master? But experts in both video game racing and artificial intelligence say the GT Sophie's success is a significant success, with agents showing a mastery of strategy and tactics.

Stanford automotive professor J. "It is a historic achievement for AI to so efficiently beat human drivers in one-on-one competition," Christian Gerdes wrote in an editorial in the scientific journal Nature. "The success of GT Sophie on the track shows that neural networks may one day play a bigger role in software for automated vehicles than they are today."

GT Sophie was trained using a method known as reinforcement learning: essentially a form of trial-and-error in which an AI agent is thrown into an environment without instruction and targets certain targets. targets. Completion is rewarded. In the case of the GT Sophie, Sony researchers say they had to design this "reward function" extremely carefully: fines to shape driving style for collisions, for example, to win. It was aggressive enough but it didn't prompt the AI to threaten other racers on the street.

Using reinforcement learning, GT Sophie was able to navigate around a racetrack with only a few hours of training and was faster than 95 percent of the drivers in its training dataset "within a day or two." After a total of 45,000 hours of training, the GT Sophie was able to achieve superhuman performance on three tracks. (For Gran Turismo Sport players, the tracks in question were Dragon Trail Seaside, Lago Maggiore GP, and Circuit de la Sarth.)

A common concern when testing AI agents against humans is that the machines have several innate advantages, such as accurate recall and faster reaction times. Sony researchers note that the GT Sophie has some advantages compared to human players, such as an accurate map of the course with coordinates of track boundaries and "accurate information about the load on each tire, the slip angle of each tire and other vehicles." states." But, he says, they account for two particularly important factors: action frequency and reaction time.

The GT Sophie's input was capped at 10 Hz, compared to the theoretical maximum human input of 60 Hz. This sometimes prompts human drivers to perform "more smooth actions" at higher speeds, the researchers wrote. As for reaction times, the GT Sophie was able to respond to events in the sporting environment in 23–30 ms, which is much faster than the estimated peak reaction times for professional athletes of 200–250 ms. To compensate, the researchers added artificial delays, training the GT Sophie with response times of 100 ms, 200 ms and 250ms. But as he found it: "All three of these tests achieved a superhuman lap time."

The GT Sophie was tested against a trio of top e-sport drivers: Emily Jones, Valerio Gallo and Igor Fraga. Although no humans outperformed the AI in the time trial, their match-ups inspired them to discover new strategies.

"It was really interesting where the AI would go, there were some corners where I was going wide and then going back in, and the AI was going around, so I learned a lot about lines, e-sports driver Emily Jones said in a testimonial in the Nature paper. "Going into Turn 1, for example, I was braking later than the AI, but the AI would get a better exit than me and beat me to the next corner. I didn't notice that until I looked at the AI and was like, 'Okay, I should do this instead.'"

Sony says it's currently working on integrating the GT Sophie into future Gran Turismo titles, but didn't give a schedule for when that might happen.