DC Networks for Generative AI
summary
Built a 24K cluster in 2023 for Llama 3 and a multi‑building 100K+ cluster (38 AI zones, >100,000 GPUs) that trains Llama 4; uses a two‑stage AI‑zone ToR/RTSW→CTSW design with 76 ATSW orthogonal planes, 144 uplinks per plane and up to ~3 km fiber spans. Faced PFC head‑of‑line blocking, lossy‑queue drops of control packets and spine buffer contention; fixes include deep‑buffer CTSWs, aggressive RTSW thresholds, moving control traffic to the lossless queue, vendor transient‑PFC mitigation and firmware/monitoring patches; design scales but Prometheus aims for 100+ km and ~10^5–10^6 GPUs.
AI-written summary · about
chapters (10)
- 0:00Introduction and motivation
- 1:07AI zone building block
- 2:46Agenda and network evolution
- 3:3824K cluster architecture
- 5:05100K cluster design
- 8:20Regional connectivity
- 10:03Running massive workloads
- 11:41Scaling challenges
- 14:02Production issues and fixes
- 18:02Conclusion and future plan
related talks
27mFaster Than Fast: Networking and Communication Optimizations for Llama 3
17mMeta’s Network Journey to Enable AI
22mGenAI Training In Production: Software, Hardware & Network Considerations
23mNetworking for GenAI Training and Inference Clusters
21mDesigning Scalable Networks for Large AI Clusters: Challenges and Key Insights
17mTraffic Engineering for AI Training Networks
22mResearch SuperCluster(RSC): Accelerating AI Research at Scale
06mOpening Remarks