Building a level 5 autonomous vehicle (in TrackMania)
summary
Reinforcement learning agent learns to race TrackMania using a custom OpenAI Gym wrapper, simulated keypresses and training on a 2017 Windows PC with a GTX 1080 Ti. Pipeline uses stable-baselines3 with PPO and later DQN; training phases go from screenshot-only to adding speed/steering, simple ray-based crash detection (threshold ~420 pixels), then hooking the OpenPlanet game API via sockets. Track times improve: ~49s at 10h, ~40s phase1, ~30s phase2, ~20.5s phase3 (human ~18s). A heavy OpenCV + DBSCAN + concave-hull approach is scrapped for being too slow; reward shaping causes exploitation of grass when API removes noisy start/finish data.
AI-written summary · about
chapters (9)
- 0:00Introduction
- 7:02AI and reinforcement learning basics
- 15:58AI training tools
- 27:21Reinforcement learning challenges
- 33:20Adding speed and turning
- 36:10Improving state representation
- 44:52Switching to DQN and crash detection
- 47:50Integrating game API
- 51:07Results and future work
related talks
22mDeveloping AI in Racing Games Using Reinforcement Learning
28mBen Lau, Quantitative Researcher, Hobbyist, NYC 2017
48mMachine Takeover - Improving Your Game With Machine Learning & AI
30mBuilding, designing, teaching and training simulation environments for Machine Learning
42mFrom Zero to Machine Learning in 45 Minutes
30mShift your ML model into overdrive with AWS DeepRacer analysis tools
01h 01mReinforcement Learning: Pac-Man
37mAWS DeepRacer: Train, Evaluate, and Tune