Cloud TPU Pods: AI Supercomputing for Large Machine Learning Problems
summary
Cloud TPUs use a custom bfloat16 format and a 128x128 systolic array to pack many 16-bit multiply‑accumulators into efficient chips; TPU v2 pods reach 11.6 petaflops at 512 cores while v3 pods hit >100 petaflops at 2,048 cores. A live demo trains RetinaNet-50 on a 300,000-image wildlife set with batch size 64×128 and finishes in 19m18s on a 128-core TPU v3 pod. Model- and data-parallel scaling is handled by XLA and Mesh TensorFlow, with 32-core pods starting at $24–$32/hr.
AI-written summary · about
chapters (25)
- 0:00Introduction
- 1:52GPU V2
- 8:16Tensorflow
- 9:33Chaos
- 11:40Models
- 12:25Object Detection
- 14:25Google Cloud Platform Notebook
- 19:21The Interconnect
- 20:01Pricing
- 20:42Cost
- 21:08eBay
- 21:33Accuracy Boost
- 22:31Data Power
- 23:26Scalability
- 24:13Complexities
- 25:40Model Parallel
- 26:24Masters of Law
- 26:51Magenta Fraud
- 29:01Training Transformer Model
- 30:05Anjana
- 33:18Demo
- 36:24Feeds
- 36:39Training Time
- 38:39Learning Rate Schedule
- 40:14Summary
related talks
06mCloud TPU v4: Fast, flexible, and easy-to-use ML accelerators
12mAI-powered infrastructure:
04mState-of-the-art Machine Learning research with Google tools
23mPyTorch on Google Cloud TPUs
28mSupercharge your training on TPUs
26mEverything Everywhere all at Once: vLLM...
23mTorchTPU: Expanding TPU Programmabil...
26mKeynote: Artificial Innovation Accelerates Artificial Intelligence