@Scale: Networking 2025
who it's for
network engineers, machine learning infrastructure engineers, data center architects, cloud infrastructure engineers, platform engineers, performance engineers, site reliability engineers
topics
RDMA over Converged Ethernet; transparent multi-NIC routing; disaggregated schedule fabric; symmetric GPU memory; zero-copy RDMA transport; alltoall and collective optimizations; PFC, ECN and DCQCN tuning; programmable NIC overlays (ASAP2/PSP); fine-grained training observability (Minder/Minecraft); scaling Llama4 to 100k GPUs; backbone and outside-plant fiber scaling; multi-tenant GPU data-center design
| title | author | length |
|---|---|---|
| Event Welcome | Vignesh Vijayanath | 04m |
Conference opens with a keynote in the ballroom then splits into two tracks—ballroom and upstairs theater—with roughly 10-minute breaks allowing track… | ||
| Keynote Welcome | Gaya Nagarajan | 20m |
Meta describes building an LLM-based outage tool Sevmate that identifies the top five suspect code check-ins to cut outage time, and reports deploying two 24K… | ||
| DC Networks for Generative AI | Rohit Puri, Hany Morsy | 21m |
Built a 24K cluster in 2023 for Llama 3 and a multi‑building 100K+ cluster (38 AI zones, >100,000 GPUs) that trains Llama 4; uses a two‑stage AI‑zone… | ||
| RDMA at Cloud Scale: The OCI Experience | Jag Brar, David Becker | 20m |
Oracle OCI describes building production RDMA over Converged Ethernet (RoCE) fabrics that scale from 1,000 to 100,000 nodes, moving from two-tier to three-tier… | ||
| Scaling AI Network with DSF | Ron He, Ankur Singh | 19m |
A credit based disaggregated schedule fabric converts packets into fixed-size cells, sprays them across fabric cards, and reassembles at egress to avoid… | ||
| Transparent MultiNIC Routing for Large AI Models | Yingjie Gu, Takshak Chahande | 22m |
Meta describes a host-level system that exposes each NIC to containers with per-NIC task IPs, uses an eBPF kernel hook to ECMP outbound flows and installs a… | ||
| Architecting Multi-tenant Data-center Networks for GPU Customers | Chang Kim, Weilong Cui | 18m |
Google’s A4 (B200) pods virtualize the GPU backend by programming CX7 NICs’ ASAP2 programmable block via the Docaflow SDK to run a PSP-based overlay. Measured… | ||
| Inference Deployments and Comms Implication | Cen Zhao, Xiaodong Wang, Jianyu Huang | 22m |
Introduces DDA flat and DDA tree all‑reduce variants that trade extra data movement for fewer steps and report up to 50% speedup on small messages (<5 MB)… | ||
| Enhancing Runtime Reliability in LLM Training via Fine-Grained Observability | Lei Zhang | 21m |
Large-scale LLM training runs on tens of thousands of GPUs for days or weeks and is fragile to tiny faults. The team builds Minder, an unsupervised VAE+LSTM… | ||
| Symmetric Memory: A New Paradigm for Programming Distributed AI | Natalia Gimelshein | 19m |
PyTorch symmetric memory implements a one-sided shared-memory API (prototype landed in PyTorch) that lets GPUs read/write peer memory to run low-latency… | ||
| Performance Optimizations at 100K+ Scale | Ashmitha Jeevaraj Shetty, Min Si | 22m |
Meta scales Llama 4 training to nearly two trillion parameters on over 100k GPUs by reworking topology (oversubscription cut from 1:7 to ~1:2.7 across… | ||
| Scaling Llama4 Training to 100K | Saif Hasan | 22m |
Claims scaling Llama4 training to a 100k‑GPU class cluster and raising effective GPU utilization from ~50% to ~90%. Key wins: replacing root init with TCP… | ||
| 10x Backbone: Scaling Backbone Connectivity to Serve AI Demands | Mark McKillop, Alberto Herrero Mediavilla | 22m |
Meta pulls backbone buildout plans from 2028 into 2024–2025 to meet AI-driven demand, using four tactics: prebuilt metro rings for fast turnups, scaling up… | ||
| Keynote | Pradeep Sindhu | 25m |
AI training needs networks that supply 200 Gbit/s per lane and roughly 100× the bandwidth per GPU versus CPU. Flat high‑radix (256–512) any‑to‑any topologies… | ||
| Live Technology Panel | Omar Baldonado | 35m |
Companies describe concrete pivots: a Tomahawk 6C 100‑Tb device to collapse tiers, a Tomahawk Ultra Ethernet switch claiming ~250 ns latency, Jericho silicon… | ||
| Live Q&A Session #1 – Srilakshmi Adusumali | Srilakshmi Adusumali | 22m |
Optical link flaps are the common failure mode and teams deploy debouncing, DSP/vendor fixes, proactive monitoring and automatic removal so repeated offenders… | ||
| Live Q&A Session #2 – Shashi Gandham | Shashi Gandham | 13m |
Moving storage into the GPU fabric creates concrete issues: implementing GPU Direct Storage to bypass CPUs, coordinating RDMA paths, and upgrading… | ||
| Live Q&A Session #1 – James Zeng | James Zeng | 22m |
Shows that tensor parallelism on a single host is practical when high‑bandwidth links like MVLink or AMD XDMI exist, but moving to RDMA across hosts forces… | ||
| Live Q&A Session #2 – Adi Gangadi | Adi Gangadi | 21m |
Panelists describe concrete fixes: network metrics (PFC, ECN, buffer and port utilization) reveal RX pauses that hampered zero-copy alltoall; switching from… | ||
| nothing matches | ||