Crowd dynamics, RL and Unity: A Journey
summary
Claims that reinforcement learning with a Kalman‑based reward produces usable crowd agents, implemented with PPO in Unity-ML-Agents (PyTorch) and deployed via Barracuda/ONNX. Training uses an entropy/statistical similarity reward derived from an ensemble Kalman filter, with experiments on only 20–30 trajectories and training runs of about 1M steps ≈7 hours. Built agents navigate new scenes, avoid collisions, and capture characteristic behaviors after data augmentation and curriculum learning, though results are approximate rather than perfect.
AI-written summary · about
chapters (13)
- 0:00Intro
- 1:16THE PROBLEM
- 4:09PEOPLE ARE SUMMATIONS OF SOCIALITY FORCES - 1990
- 6:21REINFORCEMENT LEARNING
- 7:32THE DIFFICULTIES
- 11:35WHAT IS PPOT
- 15:38UNITY ML-AGENTS
- 17:29ONE CODE EXAMPLE
- 18:58KALMAN'S GOAL
- 23:29KALMAN FILTER DEMO
- 27:16WHY DID I TELL YOU THAT
- 32:29FINAL DEMO
- 33:46CLOSING THOUGHTS
related talks
37mA practical perspective on training reinforcement learning bots.
32mThe Road to General AI
10mMachine Learning & Unity ML-Agents update
01h 17mTraining intelligent game agents using deep reinforcement learning
59mExtended Machine Learning
36mImplementing Machine Learning the Unity way
31mLayla El Asri, Research Scientist, Maluuba, NYC 2017
39mTF-Agents: A Flexible Reinforcement Learning Library for TensorFlow