Live Q+A 3
summary
Cloud uses H series and ND series VMs for CPU vs GPU workloads; rank assignment happens at application initialization using Mast topological sorting; tensor parallelism moves a few hundred megabytes per exchange so tensor parallelism is throughput bound with acceptable RTTs of a few milliseconds; traffic engineering fails to scale beyond two‑layer Clos due to operational complexity; measured GPU utilization is ~40% and improves via 4D parallelism, comm/compute overlap, and zero‑copy collectives; SuperBench is extensible via YAML.
AI-written summary · about
chapters (15)
- 0:00Introduction
- 0:27HPC vs AI hardware differences
- 1:28Tensor parallel communication
- 2:33Static rank assignment
- 3:57Traffic engineering limits
- 6:24Symmetric network state
- 7:51Scheduling constraints
- 11:28Network sensitivity
- 13:39GPU efficiency opportunities
- 17:29NIC vs fabric routing
- 19:34Optimizing data copies
- 23:27Context parallelism limits
- 25:16Automated ML scheduling
- 29:49SuperBench for testing
- 30:33Closing
related talks
13mLive Q&A Session #2 – Shashi Gandham
22mLive Q&A Session #1 – James Zeng
15mLive Q&A with Speakers – Joseph Provine
18mArchitecting Multi-tenant Data-center Networks for GPU Customers
39mScalable Multi-Node AI Workloads in Multi-Tenant AI Clouds U...
07mWelcome Remarks
23mLive Q&A with Speakers – James Zeng
17mMeta’s Network Journey to Enable AI