This project is still under heavy development. This project aims to develop a 3D simulator for learning-based methods, including reinforcement learning and imitation learning, on the F1TENTH platform. The current F1TENTH gym is a 2D environment similar to the OpenAI CarRacing Gym. The physics simulation and perception are limited in the current 2D gym environment since it can only simulate planar motion and planar LiDAR. By leveraging the rendering and dynamics in Unreal Engine 4, the new 3D simulator provides more realistic physics and better visual fidelity to reduce the Sim2Real gap further. Combining with sensors in CARLA, it opens up more potential choices of observations for training policies on the F1TENTH platform in simulation, including stereo camera, 3D LiDAR, and radar.
Here is a video of the simulator: