Maximizing GB200 and GB300 Performance with Domain-Aware Scheduling
summary
Ray adds a placement-group field called topology_strategy that enforces rack-level locality (strict_pack_on_racks) so bundles bin-pack into a single GB200/GB300 rack rather than scattering across racks. Benchmarks show MVLink+SHARP yields 920 GB/s all‑reduce vs 338 GB/s (a 2.7x speedup), and a Groot M1.7 pretraining experiment on 512 GB300 GPUs (128 nodes) gets a 14.4% run speedup when constrained to eight racks. The scheduler auto‑reschedules failed nodes, can migrate to spare racks, and integrates with Ray/Kubernetes tooling.
AI-written summary · about
chapters (9)
- 0:00Introduction and speakers
- 1:33What are GB200 and GB300
- 4:43Limitations of current Ray scheduling
- 8:43Scheduling scenarios for GB systems
- 12:13Topology-aware scheduling in Ray
- 15:04Testing with Groot training runs
- 16:51Cost and performance impact
- 17:13Key takeaways and future plans
- 21:13Q&A session
related talks
24mImproving GPU Utilization and Accelerating Model Training with Scheduling Framework and NRI
30mBeyond the DC Walls: Building a Nationwide GPU Fabric With KubeVirt and Wide-Area…
38mExploring Distributed Caching for Faster GPU Training with NVMe, GDS, and RDMA
07mLightning Talk: Environmentally Sustainable AI via Power-Aware Batch Scheduling
01h 03mExtreme Performance Architecture
21mHow Ray and Anyscale Make it Easy to do Massive-scale ML on Aerial Imagery
31mMore Nodes, More Problems: Solving Multi-Host GPU/TPU Scheduling With Dynamic Resource Allocation
32mScaling Kubernetes Clusters for Generative Models: Managing GPU Resources for AI App...