A Deep Dive into Ray’s Scheduling Policy
summary
Ray implements a distributed dynamic scheduler with a centralized GCS and per-node raylet processes and claims scalable execution of hundreds of thousands of tasks per second. Key, implemented optimizations include leasing (worker reuse) to avoid repeated five‑step worker lookup, spillback-based iterative querying for placement, and locality-aware leasing that picks the node holding most input objects to cut network transfer. Example: parallel per‑batch gradient tasks plus a combine stage — leasing and locality reduce latency and cross‑node transfers and demonstrably improve throughput.
AI-written summary · about
chapters (10)
- 0:00Introduction to Ray
- 2:17Distributed system context
- 6:21Ray scheduler overview
- 7:13The scheduling problem
- 9:51Architecture and example
- 13:35Task execution process
- 15:42Leasing optimization
- 17:27Scheduling policy
- 21:38Data locality awareness
- 25:03Summary and takeaways
related talks
26mUsing Ray On Large-scale Applications at Ant Group
27mEvolving and Scaling Ray for the Next Generation of AI Workloads
37mEvolving Ray Core: Scalability, Reliability, and Compiled Graphs
26mRay: A General Purpose Serverless Substrate?
29mAdopting Ray @ LinkedIn
29mDask-on-Ray: Using Dask and Ray to Analyze Petabytes of Remote Sensing Data
24mRay: A System for High-performance, Distributed ML Applications
19mScaling Frontier AI with Ray on Kubernetes