Keynote
summary
AI training needs networks that supply 200 Gbit/s per lane and roughly 100× the bandwidth per GPU versus CPU. Flat high‑radix (256–512) any‑to‑any topologies with two tiers scale to ~30,000 GPUs today and to ~500,000 GPUs as 200–400 Tb/s switches arrive, but power (~3 kW/GPU) becomes the limit. Copper only reaches ~2 m at 200 Gb/s, CMOS limits per‑lane speed, and a silicon implementation of a reliable packet transport shows few hundred nanoseconds jitter at high load.
AI-written summary · about
chapters (13)
- 0:00AI interconnect challenges
- 1:09AI networking differences
- 3:58AI network topology basics
- 6:54Scale-up topology design
- 8:19Scale-out topology design
- 9:47Physical layer fundamentals
- 11:54Endpoints and medium trade-offs
- 14:20Logical layer for AI workloads
- 17:06Network design principles
- 18:12Straw man AI interconnect
- 19:59Implementation details
- 22:30Performance results
- 23:54Conclusion
related talks
12mKeynote: AI Infrastructure
17mMeta’s Network Journey to Enable AI
21mDesigning Scalable Networks for Large AI Clusters: Challenges and Key Insights
23mNetworking for GenAI Training and Inference Clusters
19mAI/ML Networking Challenges: The Fast and the Finicky!
39mScalable Multi-Node AI Workloads in Multi-Tenant AI Clouds U...
10mKeynote Analysis
09mAI for NI