Improving Ray for Large-scale Applications
summary
Ant Group runs Ray at >200,000 CPU cores and reports concrete changes: replaces relay actor RPC with direct actor calls via gRPC implemented in C++ for a 6x latency improvement and 10x RPC throughput; moves GCS to a GCS service with pluggable storage to guarantee actor restart and restore state, achieving ~1.5s per actor and ~70s for 10,000 restarts; adds dashboard, fast job submission, multi‑tenancy, supports 1,000+ nodes/10,000 actors today and targets 5,000 nodes/50,000 actors.
AI-written summary · about
chapters (17)
- 0:00Introduction
- 0:29Overview
- 4:03Rearchitecture
- 4:54Performance
- 6:11Direct Actor Course
- 7:10PC Optimizations
- 7:49Active Failure Recovery
- 9:50Centralized Actors
- 10:57GCS Service
- 12:51New Challenges
- 13:56How to fix occasional errors
- 15:41How to support more workloads
- 16:43How to deploy
- 17:41Debugability
- 18:34Ray Dashboard
- 19:57Job Submission
- 20:49Roadmap
related talks
27mEvolving and Scaling Ray for the Next Generation of AI Workloads
26mUsing Ray On Large-scale Applications at Ant Group
14mKeynote: The Future of Ray
37mEvolving Ray Core: Scalability, Reliability, and Compiled Graphs
15mRay: A Distributed Compute Engine for AI
28mRay and MAI-Thinking-1
24mDistributed Black-Box Model Explanation with Ray
26mRay: A General Purpose Serverless Substrate?