Keynote: The Physics of Thought and the Architecture of Intelligence
summary
Maps historical 5x efficiency breaks to modern models and hardware, claims transformers delivered ~5x compute efficiency over LSTMs, and argues specialization drives >100x system gains. Describes Google TPU generations up to TPU v8, a rack and pod design with 9,600‑chip pods and fabric scaling to 47 Pabits and federation beyond 134,000 TPUs. Highlights use of optical circuit switches to hot‑swap racks, halve latency with a dragonfly/torus variant, and raise real goodput versus brittle theoretical peak.
AI-written summary · about
chapters (8)
- 0:00Introduction
- 3:54Industrial revolution to AI era
- 13:32Mind multiplier and intelligence
- 23:23AI demands and architecture
- 31:36Hardware foundation for AI
- 40:22Reliability and scale in AI
- 48:11Software orchestration and performance
- 52:27Future of agents and responsible AI
related talks
43mKeynote: Tiny Data Collectors, Vastly (...) & the Land of Tiny Challenges
10mKeynote: How to Leverage PyTorch to Scale AI Training and Inferencing
14mFine-tuning giant neural networks on commodity hardware with automatic pipeline...
11mLightning Talk: Tensor and 2D Parallelism
34mPanel Discussion: Training and Inference at Planet Scale
16m[Opening Keynote] Making Distributed Computing Easy
38mMPI Meets Machine Learning: Unlocking PyTorch distributed for scaling AI workloads
35mBuilding a 5-Exaflop Supercomputer for Meta-AI Research and...