@Scale: Systems & Reliability 2025
who it's for
machine learning infrastructure engineers, site reliability engineers, platform engineers, GPU cluster operators, engineering managers, storage engineers, data engineers, performance engineers
topics
gang scheduling for MoE Llama models; distributed inference model partitioning; context parallelism for LLM prefill; large GPU cluster operations; model freshness and deployment pipelines; ultra-low latency LLM inference; QLC flash storage optimizations; global service placer for region abstraction; KV store design for ML workloads; model registry and stability metrics; silent data corruption detection; hybrid parallelism and hardware co-design
| title | author | length |
|---|---|---|
| Gang Scheduling for Llama | Anca Agape, Andre Darabanov | 24m |
Llama 4 uses a MoE design with >2 trillion parameters but only activates ~17B for Maverick, forcing distributed inference. Twine implements gang scheduling to… | ||
| Live Keynote | Surupa Biswas, Peter Hoose | 29m |
Meta describes rapid infrastructure changes: a virtual machine vending machine built in months runs 250,000 concurrent VMs and reports 40 billion+ code… | ||
| Live Fireside Chat | Jay Parikh, Francois Richard | 23m |
Conference founded around 2012–13 with three concrete rules: focus on scale engineering, for engineers by engineers, and open community. Hard lessons: expect… | ||
| Opening Remarks | Francois Richard | 05m |
Conference opener claims systems now manage hundreds of thousands of containers across continents and layer AI/ML workloads with different failure modes… | ||
| The Need for Speed: The Story of How We Achieved Industry Leading TTP | Richard Wareing, Tyler Graf | 21m |
Meta resets NPI scope to include the full software stack (schedulers, frameworks, customer models) and gives product groups a veto on PVT so production… | ||
| Model Freshness and Its Infra Implications | Vivek Khurana, Xianzheng Dou, Lujia Zhang | 20m |
Meta recounts concrete infra changes that cut model update time: moving snapshot publishing into training, aggressive snapshot compression, and a network-local… | ||
| AI Hardware Reliability at Scale | Sriram Sankar, Harish Dixit | 21m |
At Meta over 66% of training job interruptions come from hardware failures and silent data corruptions occur at roughly one in a thousand devices versus… | ||
| Journey to 1000 Models: Scaling Instagram's algorithm without the Reliability Nightmare | Sing Sing Ma, Luke Levis | 19m |
Instagram scales from O(10) GPU models to over 1,000 in production and replaces manual release and monitoring with a model registry that maps numeric IDs to… | ||
| From Silicon to Scale: Training at Scale | Zhenyu Gu | 13m |
AMD shows a concrete stack and outcomes: full-stack hardware/software co‑design using rocm, PyTorch, Triton and Hugging Face, hybrid parallelism (tensor +… | ||
| How We’re Scaling Discovery at Netflix Reliably | Karthik Puthraya, Saurabh Jaluka | 19m |
Mosaic constructs Netflix member pages for ~300 million households and >300 client surfaces and faces a combinatorial explosion of server driven… | ||
| Splitting the Monolith | Phil Lopreiato, Zach Zundel | 20m |
Meta converts its single monolith to a dedicated generative AI tenant that keeps the same developer surface but changes the infra: a smaller colocated… | ||
| Advancing Flash Storage | Sumit Gupta, Riley Thomasson | 12m |
Meta deploys high‑density QLC racks with a co‑designed solution from Pure Storage that pushes SSD smarts into direct flash software and simple DFMs to hit… | ||
| A Planet-Scale Computer – Abstract Away Regions via Global Service Placer (GSP) | Gerald Guo, Yunqi Zhang, Yatpang Cheung, Yan Yan | 18m |
Meta builds a system that autonomously places services across regions using a placement planner and executor, formulating placement as a constrained bin… | ||
| Building KVStore for ML Workloads | Jia Zhan | 19m |
Pinterest runs KVStore, a C++ distributed key-value store built on RosDB with Helix shard management, ingesting bulk files from S3 and streaming from Kafka or… | ||
| Challenges with Ultra-low Latency LLM Inference at Scale | Haytham Abuelfutuh | 24m |
Union claims an inference service that outperforms managed offerings — “multiples faster” than SageMaker and up to 5× faster than some APIs — by doing three… | ||
| Turbocharging AI/ML workloads: Revving Up Speed and Resilience | Lerna Ekmekcioglu | 20m |
Demonstrates that lack of Q-pair visibility causes undiagnosed slow flows and can cut throughput by up to 50%, shows live Oracle Cloud demos measuring one‑way… | ||
| Experience Operating Large GPU Clusters at Organizational Scale | Vipin Sirohi, Mohamed Fawzy, Bugra Gedik, Vikas Mehta | 21m |
Nvidia describes operating multi-tenant GPU clusters at ~95% occupancy by combining multi-level fair sharing (account, quota, user, workload) with minimum… | ||
| Product Reliability in | Micah Lerner | 18m |
Shift from server signals to instrumentation of UI journeys, tracing with Proto on Android, and automated availability/latency/data quality checks tied into… | ||
| nothing matches | ||