FAST: An Efficient Scheduler for All-to-All GPU Communication
summary
FAST reframes All-to-All(v) scheduling by treating scale-up (NVLink/local GPU) as effectively free and optimizing only scale-out transfers, implementing Birkhoff decomposition to break traffic into balanced, incast-free one-to-one stages and using intra-server rebalancing to shrink the largest row/column loads. Implementation on NVIDIA H200 and AMD MI300X clusters shows 20–30% higher bandwidth than state-of-the-art and computes schedules for 300 GPUs under 100 ms (microsecond-to-millisecond typical), delivering practical, faster transfers.
AI-written summary · about
chapters (12)
- 0:00FAST scheduler intro
- 1:22All-to-all communication importance
- 2:25Optimizing all-reduce challenges
- 3:45Why scheduling is a hard problem
- 5:31Waypointing in GPU clusters
- 7:14Scale-up efficiency insight
- 8:41Global scheduling with Birkhoff
- 11:11Local scheduling for bottlenecks
- 14:01Sender-receiver load balance
- 15:23Pipelining scale-up and scale-out
- 16:05Nvidia and AMD performance
- 17:07Summary and open-source
related talks
31mShared GPU Scheduling & Proactive Autoscaling: A Production Blueprint for 1000+…
17mServing Heterogeneous Machine Learning Models on Multi-GPU Servers...
17mEnabling Efficient GPU Communication over Multiple NICs with FuseLink
20mScheduler and Sharding Considerations for Network Efficiency
41mBeyond Uptime: Revolutionizing Fintech Reliability
36mAccelerating AI training fleets with sched_ext
13mARK: GPU-driven Code Execution for Distributed Deep Learning
14mBlitzScale: Fast and Live Large Model Autoscaling with O(1) Host Caching