Evolving Ray Core: Scalability, Reliability, and Compiled Graphs
summary
Weekly releases and structured logging reduce release latency and improve observability, streaming generators let callers consume object refs while tasks still run, and cluster scaling work pushes Ray Core to 8,000 nodes. A new backend, Ray Compiled Graphs, compiles static task graphs, allocates reusable object-store buffers, declares peer‑to‑peer GPU edges (transport hints on torch tensors), schedules NCCL/NVLink operations to avoid deadlock, and overlaps comms with compute. Benchmarks report 17x faster single‑node and ~2.8x faster multi‑node execution for static graphs; beta coming soon.
AI-written summary · about
chapters (10)
- 0:00Introduction to Ray Core
- 1:06Ray Core overview
- 2:12New Ray Core features
- 5:22Why compiled graphs
- 7:31Compiled graphs API
- 12:47GPU programming with compiled graphs
- 18:29Use cases for compiled graphs
- 22:50Future of compiled graphs
- 23:54Virtual clusters in Ray Core
- 25:18Q&A session
related talks
27mEvolving and Scaling Ray for the Next Generation of AI Workloads
14mKeynote: The Future of Ray
28mA Deep Dive into Ray’s Scheduling Policy
23mImproving Ray for Large-scale Applications
26mRay: A General Purpose Serverless Substrate?
22mDebugging and Observability for Distributed Ray Applications
11mKeynote Q&A
15mRay: A Distributed Compute Engine for AI