Large Scale Data Loading and Data Preprocessing with Ray
summary
NVIDIA describes replacing PyTorch’s multiprocessing DataLoader with Ray actors and the Ray object store to run dataset get_item calls in separate processes, keeping existing sampler/dataset code. The Ray pipeline matches PyTorch DataLoader same throughput while eliminating intermittent deadlocks. Production deployment requires strict MPI-style Ray init/teardown order for multi‑GPU jobs and tuning (worker_register_timeout_seconds, heartbeat timeouts) on Kubernetes. The system runs reliably at scale.
AI-written summary · about
chapters (9)
- 0:00Introduction
- 0:40Agenda
- 1:35Data Loading
- 6:26Python Data Loader
- 10:07Python MultiProcess Engine
- 12:02Why use Ray
- 14:35Sketch
- 16:00Benefits
- 19:06Lessons Learned
related talks
33mOvercoming Distributed ML Challenges with Ray Train
35mOperating a Production-Scale Physical AI Data Platform with Ray
28mScaling Multimodal Data and Reinforcement Learning with Ray
30mHow to Scale Unstructured Data Processing with Ray Data
24mRay: A System for High-performance, Distributed ML Applications
24mBuilding Complex Data Analytics Pipelines with Ray
35mHow Bytedance Builds Large-Scale Data Processing Pipelines for Multimodal Models with Ray
22mRay, a Unified Distributed Framework for the Modern AI Stack