AI is often divisive, especially outside of sim racing. However, the use of AI has been normal within racing games for decades. So, what does Sophy bring to GT7, and how is it progressing the Sony AI project?
OverTake.gg sat down with two of the very talented minds behind the Sony AI program: Kaushik Subramanian, a Senior Staff Research Scientist, and Takuma Seno, a Senior Research Scientist. Here is what they had to say about Sophy and the future of the tech for sim racers.
Image: Sony AI
GT Sophy was trained using novel deep reinforcement learning techniques, including state-of-the-art learning algorithms and training scenarios developed by Sony AI, GTSport, and leveraging SIE's cloud gaming infrastructure for massive-scale training. The agent differs from other, more conventional approaches in that it learns to race by trial and error through its own experience.
The agent is able to learn the target behaviour by training in different racing scenarios, while the conventional in-game AI agents primarily exhibit behaviour hand-coded by game designers. Because of this, GT Sophy is more reactive and robust over a diverse range of user interactions in the game." - Kaushik Subramanian
Rival setup with Sophy AI.
To cover custom races, it was important that we test and refine the AI agent on various race conditions to experiment with the different race setups that users could curate themselves. At the same time, we worked to improve GT Sophy's race craft with updated training scenarios (to cover the variety of cars under different custom race settings), and iterated on the reward functions and neural network architectures to improve the agent's robustness and satisfy the high racing standards - Takuma Seno
Sophy AI race setup screen
This new update now enables players to race against GT Sophy using over 400 cars on 19 tracks. But the most significant change is that they can modify the conditions of the race, including choosing the number of laps and cars participating in the race, customising GT Sophy's cars, and setting tire and fuel consumption rates. The agent has been trained to drive in any condition the user specifies. - Takuma Seno
The goal of this exploratory research was to see if we could train an RL agent to drive as well as top human drivers using only game images as input. Being able to learn from image data from the game is one step in generalising the capabilities of the RL agent. - Kaushik Subramanian
We believe there is always room for continued exploration regarding improving the agent's racecraft and robustness and providing racing enjoyment to the full spectrum of GT players. - Kaushik Subramanian
Sophy AI tactic tree
The AI question in sim racing has been rather divisive for a number of years, whether it's a discussion about which sim has the best drivable AI or driver coaching tools like OverTake Turbo that we just launched. AI is immensely beneficial to sim racers and the gaming world more broadly, and is Sophy heading that up, in your opinion?
Do you use Gran Turismo 7's Sophy AI? Are there any other questions you want answered? Let us know in the comments down below!
OverTake.gg sat down with two of the very talented minds behind the Sony AI program: Kaushik Subramanian, a Senior Staff Research Scientist, and Takuma Seno, a Senior Research Scientist. Here is what they had to say about Sophy and the future of the tech for sim racers.
Image: Sony AI
The GT7 Sophy Interview
To start off and fill in those who may not know yet, could you explain how GT Sophy works and how it differs from other, more conventional models?
"Gran Turismo Sophy is a breakthrough AI racing agent developed through a collaboration between Sony AI, Polyphony Digital, and Sony Interactive Entertainment. The initial research goal of GT Sophy was to create the first superhuman AI agent to outrace the world's best players in Gran Turismo Sport.GT Sophy was trained using novel deep reinforcement learning techniques, including state-of-the-art learning algorithms and training scenarios developed by Sony AI, GTSport, and leveraging SIE's cloud gaming infrastructure for massive-scale training. The agent differs from other, more conventional approaches in that it learns to race by trial and error through its own experience.
The agent is able to learn the target behaviour by training in different racing scenarios, while the conventional in-game AI agents primarily exhibit behaviour hand-coded by game designers. Because of this, GT Sophy is more reactive and robust over a diverse range of user interactions in the game." - Kaushik Subramanian
Rival setup with Sophy AI.
After two years of being available, GT Sophy has evolved to be even more flexible for racers to use in more scenarios. How difficult was it to get to this point? Was a longer time frame like this necessary to generate enough of a sample size for GT Sophy to work well with even more variables?
We are constantly looking for ways to expand the agent's capabilities. Training GT Sophy to be accessible to a broader range of players while covering many of the options available in custom races was challenging. As we deployed GT Sophy 2.0 to more tracks and continued to increase the number of cars it could drive, we gathered valuable feedback from the player community about how they'd like to interact with it.To cover custom races, it was important that we test and refine the AI agent on various race conditions to experiment with the different race setups that users could curate themselves. At the same time, we worked to improve GT Sophy's race craft with updated training scenarios (to cover the variety of cars under different custom race settings), and iterated on the reward functions and neural network architectures to improve the agent's robustness and satisfy the high racing standards - Takuma Seno
Sophy AI race setup screen
We'd imagine there would be different sample sizes for cars that are less popular to drive, so the oddballs in particular. How does the model account for this?
Our training approach focuses on being able to drive all the cars in the game equally well. GT7 has such a large variety of cars with a broad range of specifications, so there are some cars that pose a generalisation challenge to the agent. This is primarily because of how unique their setup is. Our updated training schemes allow us to identify the cars that the agent has difficulty with and adaptively vary how often these cars are selected to give the agent more targeted data to learn from. - Takuma SenoWhat are the main differences between Sophy 2.1 and what had been available before GT7's 1.57 update?
With GT Sophy 2.1, players have greater control over their racing experience than they did in prior releases. When GT Sophy 2.0 debuted as a full in-game feature in November 2023, players were able to go head-to-head against the GT Sophy using more than 340 cars on nine different tracks, a significant step up from the four tracks and four player car choices in the limited-time release in February 2023.This new update now enables players to race against GT Sophy using over 400 cars on 19 tracks. But the most significant change is that they can modify the conditions of the race, including choosing the number of laps and cars participating in the race, customising GT Sophy's cars, and setting tire and fuel consumption rates. The agent has been trained to drive in any condition the user specifies. - Takuma Seno
Could an advanced technology like GT Sophy be used in real-life scenarios as well, such as cars driving or racing autonomously?
We are always exploring new possibilities for GT Sophy and cannot comment on anything further regarding the agent's functionality outside of the GT Sophy 2.1 update. However, as part of Sony AI's continued effort to explore interesting research challenges in Game AI, last summer, we released exploratory research that introduced a super-human racing agent whose sensor input is purely local to the car.The goal of this exploratory research was to see if we could train an RL agent to drive as well as top human drivers using only game images as input. Being able to learn from image data from the game is one step in generalising the capabilities of the RL agent. - Kaushik Subramanian
Is there an end goal for GT Sophy, or is it set to be an ever-evolving model?
A high-fidelity simulator like Gran Turismo offers several interesting research opportunities, both from expanding what GT Sophy can do in the game and pushing the limits of what deep reinforcement learning agents are capable of. We are constantly exploring new possibilities for GT Sophy and enjoy receiving feedback from the GT players so we can continue testing and refining the AI agent's capabilities.We believe there is always room for continued exploration regarding improving the agent's racecraft and robustness and providing racing enjoyment to the full spectrum of GT players. - Kaushik Subramanian
Sophy AI tactic tree
Does GT Sophy have the capability to recognise if it has been overly aggressive? In turn, can it recognise if the player has been and look for "payback"?
We work to ensure that the agent always performs at its best. In the training process, we can determine how aggressively the agent behaves and make changes to correct it if necessary. When deployed in the game, GT Sophy responds to instantaneous signals in racing situations; it does not incorporate memory or real-time adaptation. - Kaushik SubramanianAI in Sim Racing
With these answers, the future of the Sophy AI is certainly under wraps, and many implementation questions beyond the actual creation of the AI were shut down almost immediately. So, whilst some impressive progression is being shown, there is undoubtedly a lot that Sony AI is keeping a secret... for now!The AI question in sim racing has been rather divisive for a number of years, whether it's a discussion about which sim has the best drivable AI or driver coaching tools like OverTake Turbo that we just launched. AI is immensely beneficial to sim racers and the gaming world more broadly, and is Sophy heading that up, in your opinion?
Do you use Gran Turismo 7's Sophy AI? Are there any other questions you want answered? Let us know in the comments down below!
