Waymo's EMMA: Teaching Cars to Think
summary
Waymo presents EMMA, a Gemini-based end-to-end multimodal planner that ingests eight-camera video plus routing text and directly predicts future waypoints; it trains self-supervised from driving logs, is camera-only and map-free (besides routing), and matches or beats state-of-the-art on an open-loop planner benchmark. Adding intermediate outputs—critical-object detection and behavior explanations—improves performance against strong baselines that use oracle perception and HD maps, and use of generative video for sensor simulation shows clear drops in rain and night conditions.
AI-written summary · about
chapters (12)
- 0:00Introduction to autonomous driving
- 1:27Why Waymo's L4 system excels
- 2:54Scaling challenges and road trip
- 4:17Foundation models for long-tail scenarios
- 6:27Introducing EMMA for autonomous driving
- 8:16EMMA's self-supervised approach
- 9:39Improving EMMA with reasoning
- 11:46Foundation model scaling laws
- 12:49EMMA's multi-task capabilities
- 14:16Evaluating EMMA safely
- 15:19Sensor simulation for testing
- 16:42Conclusion and future work
related talks
37mBuckle Up: How Waymo Uses Data to Create a Safer Driving Experience
31mDeep learning for self-driving vehicle technology
40mTeaching a Car to Drive Itself by Imitation and Imagination
27mDriving Autonomous Vehicles Forward w/Real World Applications
41mSelf-Driving Lessons from Comma AI
05mInterview
49mUsing Deep Learning for Perception in Autonomous Systems: A Perspective from Control Theory
21mHow Google DeepMind is researching the next Frontier of AI for Gemini