Networking @Scale 2024
2024-09-11
Santa Clara, United States
[Networking @Scale 2015, Networking @Scale Summer 2022, Networking @Scale Fall 2022, Networking @Scale 2023, Networking @Scale 2024]
who it's for
machine learning infrastructure engineers, network engineers, data center architects, site reliability engineers, platform engineers, HPC engineers, observability engineers
topics
edge autoscaler for GPU clusters; GPU virtualization for multi tenancy; RDMA and InfiniBand verbs; Slingshot Ethernet deployments; smart NICs for application offload; nickx per-work-request tracing; Echo CCL and SOLAR transport; topology-aware placement for Llama 3; NVLink direct data access; enhanced ECMP with flow multiplexing; scheduler and sharding for network efficiency; per-minute loss measurement and netNORAD
| title | author | length |
|---|---|---|
| Opening Remarks | Omar Baldonado | 06m |
Opening remarks set the agenda: AI networking as an end-to-end problem spanning data center, backbone and edge; operations must scale as model training grows… | ||
| Evolving Meta's Edge Architecture | Shivkumar Chandrashekhar, Lee Hetherington | 20m |
Meta evolves a globally distributed Edge into a decentralized multi‑tenant compute platform that places GPUs and NVMe closer to users, claiming global average… | ||
| Future Challenges for HPC Networking @Scale | Ron Brightwell | 21m |
Networks shift toward commodity Ethernet with Slingshot (Ethernet) deployed on Exascale machines and the Ultra Ethernet Consortium forming, but MPI still… | ||
| AI Impact on the Backbone | Jyotsna Sundaresan, Abishek Gopalan | 20m |
Meta's backbone connects 25+ data centers and 85 PoPs over millions of miles of fiber and reports 30–50% traffic increase from AI since 2022 as GPU demand… | ||
| Live Q+A 1 | Ying Zhang | 31m |
Panel argues that traditional CDNs remain but AI shifts work to compute‑heavy edge nodes with many GPUs and less emphasis on SSD/NVMe caching; operators deploy… | ||
| Network Communication Debuggability and Observability at Scale | Ashmitha Jeevaraj Shetty, Min Si | 23m |
Meta instruments its collective library with a custom nickx communication library that adds real-time performance tracing at per-work-request granularity… | ||
| Alibaba HPN: A Data Center Network for Large Language Model Training | Jiaqi Gao | 15m |
A purpose-built 2-tier, dual-plane datacenter fabric connects segments of up to 1,000 accelerators per segment and scales to 130k accelerators using 51.2 Tbps… | ||
| High Network Reliability & Availability in FE & BE for Scalable Training Solutions | Jose Leitao, Robert Colantuoni | 21m |
Meta runs separate FE and BE fabrics, with BE clusters built from ~18 CTS spines, ~252 RTWs per AI zone and roughly 4,000 GPUs per cluster, so a single device… | ||
| Live Q+A 2 | Joseph Provine | 32m |
Operators use bandwidth expectations tied to BDP so large transfers hit about 10% lower than line rate and hop count (leaf vs two‑hop) sets per‑pair targets… | ||
| Scheduler and Sharding Considerations for Network Efficiency | Weiwei Chu, Arnab Choudhury | 20m |
Describes how MAST and topology-aware placement map Llama3 parallelisms onto a 24,000 GPU network to cut cross‑hop traffic, using rank assignment and… | ||
| Designing Scalable Networks for Large AI Clusters: Challenges and Key Insights | Jithin Jose | 21m |
Microsoft reports moving from 8K–20K HPC jobs to A100/H100 clusters used across hundreds of thousands of GPUs and claims Top500 cloud rank 1 and MLPerf cloud… | ||
| Faster Than Fast: Networking and Communication Optimizations for Llama 3 | Pavan Balaji, Adi Gangidi | 27m |
Meta builds two 24,000‑GPU clusters (AI zones of ~3,000 GPUs each, 1:7 oversubscription at aggregate switches) to train Llama 3 (8B, 70B, 405B) and reports… | ||
| Live Q+A 3 | Shashi Gandham | 31m |
Cloud uses H series and ND series VMs for CPU vs GPU workloads; rank assignment happens at application initialization using Mast topological sorting; tensor… | ||
| nothing matches | ||