Keynote: AI Infrastructure
summary
Company scales AI infra from CPU fleets to GPUs and custom silicon, runs training jobs at hundreds to low thousands of GPUs historically and now needs tens of thousands for large models, and plans to reach 600,000 GPUs. Engineering choices include a 2019 PyTorch migration, end-to-end hardware/software co‑optimization (Open Compute plus in‑house silicon), a componentized inference platform, frequent checkpointing to reduce failure cost, and separate pipelines for event ingestion (~tens of TB/s) versus large static corpora for foundational model training.
AI-written summary · about
chapters (7)
- 0:00AI infrastructure scaling
- 2:11Scaling compute for AI
- 5:44Hardware and software optimization
- 6:27Efficient AI model development
- 9:17Managing AI data at scale
- 10:42Reactive AI-powered products
- 11:44Future scaling challenges
related talks
14mKeynote: Building the Infrastructure that Powers the Future of AI
24mKeynote: Next-generation frameworks for Large-scale AI
10mKeynote: How to Leverage PyTorch to Scale AI Training and Inferencing
29mLive Keynote
06mKeynote: Intel and PyTorch: Enabling AI Everywhere with Ubiquitous Hardware and Open...
23mKeynote
28mScaling AI Infrastructure
20mOperating Frontier Intelligence at Scale