Operating a Production-Scale Physical AI Data Platform with Ray
summary
Hyundai runs Ray as the foundation of a physical AI data platform processing petabyte-scale sensor data for autonomous driving, automating model-based curation and privacy-preserving de-identification. Migration from Airflow+Triton to Ray cluster + MLflow reduces code changes and operational complexity. Techniques include torch compilation cache reuse, NFS prefetching and runAI model streamer (up to 5x faster for large TP models), RDMA to reach near-linear multi-node PyTorch DDP scaling, and preferring single‑GPU VLM pods for fault isolation; typical footprint is about 15–20 Ray clusters.
AI-written summary · about
chapters (12)
- 0:00Introduction and agenda
- 0:48GPU-intensive processing
- 3:48Lessons on Ray core
- 5:04Multi-node training
- 6:11Observability and optimization
- 7:13Standardized model serving
- 8:49Scaling VLMs and WFM
- 17:16Serving strategy and case study
- 20:35Platform architecture overview
- 22:49ArgoCD and gitops practices
- 24:00Agent-based evolution
- 27:20Q&A and closing
related talks
22mRay, a Unified Distributed Framework for the Modern AI Stack
28mScaling Multimodal Data and Reinforcement Learning with Ray
31mNext-Generation Data Science Workflows Using Ray
22mLarge Scale Data Loading and Data Preprocessing with Ray
25mData Science at Scale: From Notebook to Production with Airflow and Ray
29mScaling ML Pipelines for Autonomous Driving with Ray
28mScaling AI & Machine Learning Workloads With Ray on AWS, Kubernetes, & BERT
21mHow Ray and Anyscale Make it Easy to do Massive-scale ML on Aerial Imagery