Pinterest's ML Evolution: Distributed Training with Ray
summary
Pinterest uses Ray and Ray Data to build a distributed data loader that moves CPU-heavy file IO and preprocessing off GPU trainers, implements batching as Ray actors, and serializes each batch into a single binary buffer. A custom Zstd compression patch to the Ray object store and a serialized batch buffer eliminate per-feature unpickling and cut network bytes; adding pin-memory + CUDA stream preloading restores GPU overlap. Result: end-to-end throughput reaches ~400k examples/sec (≈3.6× the initial Ray run, 50% faster than the original PyTorch dataloader) and scales to ~32 CPU nodes.
AI-written summary · about
chapters (8)
- 0:00Introduction to Pinterest ML
- 2:20Training pipeline architecture
- 5:14Identifying the bottleneck
- 7:52Initial data optimizations
- 12:21Adopting Ray for scale
- 15:14Addressing Ray performance
- 22:47Final pipeline improvements
- 25:28Key takeaways and conclusion
related talks
44mPanel: ML Platform on Ray
22mLarge Scale Data Loading and Data Preprocessing with Ray
26mData Processing on Ray
28mScaling Multimodal Data and Reinforcement Learning with Ray
21mHow Ray and Anyscale Make it Easy to do Massive-scale ML on Aerial Imagery
22mRay, a Unified Distributed Framework for the Modern AI Stack
24mRay: A System for High-performance, Distributed ML Applications
01h 31mIntroduction to Ray for distributed and machine learning applications in Python