Keynote
summary
A tour of AI infrastructure stresses concrete numbers and engineering problems: Meta scales to 600,000 GPUs in its fleet, Llama 3.1 trains on 15 trillion tokens, and training growth runs roughly 4x per year. Responses include building custom silicon to cut cost and power, tackling networking and cooling for tens‑of‑thousands‑GPU clusters, and research on asynchronous training because a single‑GPU failure stops training today, forcing costly restarts and motivating faster failure detection and new parallelism approaches.
AI-written summary · about
chapters (25)
- 0:00Introduction
- 0:28History of AI
- 2:07AlexNet
- 2:56Model Innovation
- 4:48Data Center Computing
- 5:39Our Ambition
- 6:47Challenges
- 7:00Data
- 7:37Data bottleneck
- 8:33Compute
- 8:53Infrastructure Challenges
- 9:29Custom Silicon
- 9:46Energy Consumption
- 10:22Scaling Challenges
- 11:31Networking Challenges
- 12:05Engineering Challenges
- 12:22Model Algorithms
- 12:44Model Size
- 13:08Token Prediction
- 13:21Second School of Thought
- 14:24Generalized Intelligence
- 15:07Models
- 15:20Open Source
- 15:46The Future
- 16:12Conclusion
related talks
15mKeynote: From Platforms to AI Factories. Has Kubernetes Solved It?
22mKeynote
17mKeynote: Programming the Cloud as Easily as Your Laptop
39mKeynote: Solving the problems of AI in production
44mKeynote: Tracking the Pulse of Generative AI
59mKeynote: AI without the BS, for humans
09mAI for NI
17mKeynote: An OPEN Path to Super Intelligence