Scaling AI Infrastructure
summary
OpenAI describes concrete scaling wins and failure modes: reinforcement agents for Dota run across custom Rapid clusters with rollout, optimizer and evaluation workers using NCCL and Redis at scale; Kubernetes hits (2,500+ nodes) force anti‑affinity and etcd tuning; a migration expands capacity from 60,000 to 128,000 CPU cores with minimal researcher impact; GPU hosts require honoring a 60 minute maintenance window to avoid constant restarts; GPT-2 and MuseNet reuse ArcHall-style frameworks and served 40,000 music samples on launch.
AI-written summary · about
chapters (28)
- 0:00Introduction
- 0:51ImageNet
- 1:15Image Recognition Comic
- 1:44Neural Activation Atlas
- 6:10Supervised Learning
- 7:23Feedback Loop
- 7:58Deep Mind
- 9:25Neural Networks
- 11:54Defense of the Ancients
- 12:36Map Visibility
- 12:55Long Planning Horizon
- 13:25Learning Loop
- 13:47Kubernetes
- 15:08Rapid
- 17:16Redis
- 17:34Cloud Migration
- 18:00Maintenance Events
- 19:13DOTA
- 19:57GPTto
- 21:15Arc Hall
- 21:54Security
- 23:09Music Musenet
- 24:40One Ongoing Challenge
- 25:43Efficient Computing
- 27:04Google Kubernetes
- 27:30Recap
- 27:47Decouple experiment dependencies
- 28:02Dont apply AI training
related talks
14mKeynote: Building the Infrastructure that Powers the Future of AI
27mHarnessing Kubernetes for Scalable AI/ML Workloads
38mAI in Kubernetes: How to Get Started?
12mKeynote: AI Infrastructure
32mScaling AI: Fault-Tolerant GenAI Solutions for Millions
20mOperating Frontier Intelligence at Scale
25mHow We Power the Largest AI Deployments on the Planet: Running Vir...
01h 15mTutorial: Build, Operate, and Use a Multi-Tenant AI Cluster Based Entirely on Open Source