From Reservation to 1,600 GPUs: A Petabyte-Scale Ray Data Pipeline
summary
Anyscale and CoreWeave deploy a pipeline that turns a fresh GPU reservation into production in under 24 hours, moving from Ray Core to Ray Data and streaming execution to hit 70 million captions in 1h35m across 1,600 GPUs on a 0.6 PB video corpus. CPUs decode videos with local NVMe caching, GPUs run VLM inference, autoscaling and repartitioning fix CPU/GPU imbalance, and Ray Data’s fault tolerance replaces failed nodes so storage never becomes the bottleneck.
AI-written summary · about
chapters (10)
- 0:00Introduction to project
- 1:11Challenge and goals
- 3:15Coreweave stack overview
- 4:42Pipeline workflow explained
- 5:06Ray Data under the hood
- 6:54Scaling to 1,600 GPUs
- 8:03Identifying bottlenecks
- 10:58Optimizing throughput
- 12:01Fault tolerance at scale
- 13:30Key takeaways and summary
related talks
30mHow to Scale Unstructured Data Processing with Ray Data
32mAnyscale's Ray Data: Revolutionizing Batch Inference
35mHow Bytedance Builds Large-Scale Data Processing Pipelines for Multimodal Models with Ray
22mLarge Scale Data Loading and Data Preprocessing with Ray
20mThe Quick Journey to Using Ray: How We Implement Ray and Anyscale to Speed up our ML Processes
28mScaling Multimodal Data and Reinforcement Learning with Ray
26mData Processing on Ray
34mHow NVIDIA is Advancing Video Curation with Generative AI