The Development of a Simulated Car Racing Controller Based on Monte-Carlo Tree Search

Abstract

Ever since its introduction, Monte Carlo Tree Search (MCTS) has shown very good performances on a number of games, most of which are turn-based zero-sum games. More recently, researchers have also started to expand the application of MCTS to other types of games. This paper proposes a new framework of applying MCTS to the game of simulated car racing. We choose to build the search tree in a discretized game-state space and then determine the action from the selected target game state. This allows us to avoid the need to discretize the action space. In addition, we are able to incorporate some heuristics on driving strategies naturally. The resulting controller shows very competitive performance in the open-source racing game TORCS.