How Zoox Accelerated Autonomous Driving with Ray
summary
Zoox describes how Ray (Train, Data, Serve) plugs into an existing SLURM-backed on‑prem cluster and AWS EKS to power both multi‑node PyTorch Lightning training and a cloud inference service. Engineering choices include running persistent head nodes (one per Ray/Python combo), port‑selection to avoid SLURM co‑scheduling conflicts, and standardized launchers that produce ZTrain to uplift Lightning modules. A greenfield ZServe model‑based architecture cuts per‑model Python surface from ~500 LOC and 16 GB Docker images to about 5 lines of code per model (claimed ~100x reduction) and already serves multiple production models.
AI-written summary · about
chapters (6)
- 0:00Introduction to Zoox and Ray
- 2:35Integrating Ray with Slurm
- 7:04Standardizing ML training
- 8:59Greenfield inference projects
- 9:33Scaling with model-based ZServe
- 19:47Closing and Q&A
related talks
21mHow Ray and Anyscale Make it Easy to do Massive-scale ML on Aerial Imagery
28mScaling AI & Machine Learning Workloads With Ray on AWS, Kubernetes, & BERT
35mOperating a Production-Scale Physical AI Data Platform with Ray
44mPanel: ML Platform on Ray
28mScaling Multimodal Data and Reinforcement Learning with Ray
31mNext-Generation Data Science Workflows Using Ray
22mRay, a Unified Distributed Framework for the Modern AI Stack
32mRay in Production: Engineering Lessons from Spotify's Hendrix Framework