Keynote
summary
Claims that over 100,000 LLM projects exist and that clusters reach 24,000 GPUs set the scale then quantify failure dynamics as roughly 1.3 failures per hour. Systems built include prewarmed spare hosts and consistently fast container restarts that cut restart overhead from minutes to seconds, asynchronous checkpoints that reduce 2–4s pauses to tens of milliseconds, delta snapshots of top 10% parameters every 15 minutes, and solver-driven placement for distributed inference across heterogeneous GPU generations.
AI-written summary · about
chapters (12)
- 0:00Systems at Scale keynote
- 1:52Generative AI watershed
- 3:38Training on GPU meshes
- 5:45Hardware failures and recovery
- 8:35Optimizing training restarts
- 9:38Beyond recovery speed
- 10:20Model freshness impact
- 12:25Delta snapshots hourly
- 13:28Streaming model updates
- 14:12Inference challenges
- 15:59Distributed inference
- 16:43AI and systems innovation
related talks
29mLive Keynote
20mKeynote Welcome
12mKeynote: Build Accelerated LLM Applications with Open Source Tools from NVIDIA
42mStop Treating LLMs Like REST APIs
10mKeynote: Scaling Open Source Large Language Models
12mKeynote: AI Infrastructure- 12mKeynote: Build Accelerated LLM Applications with Open Source Tools from NVIDIA
23mKeynote