All AI Roads Lead to Distribution
summary
Claims that distribution both cuts training time and enables larger experiments with concrete numbers and tools: ImageNet is ~300 GB and 1M images, Google and Facebook push accuracy with neural architecture search and billion‑image pretraining, Facebook reports reducing ImageNet training from two weeks to one hour using 256 GPUs, Google reports up to 140,000 files/sec, and Hops demonstrates scalable storage and I/O at 1.2 million ops/sec while exposing Python APIs for data‑parallel training and AutoML workflows.
AI-written summary · about
chapters (18)
- 0:00Start
- 0:33Massive Increase in Compute for AI
- 2:04Model Parallelism
- 4:08Improvements in ImageNet - Accuracy Top-1 Accuracy for ImageNet
- 5:12Methods for Improving Model Accuracy
- 5:40Improvements in ImageNet - Training Time
- 6:06ImageNet - Files/Sec Processed
- 6:28Remove Bottleneck and keep Scaling Distributed Filesystem Read Throughout
- 7:00All AI Roads Lead to Distribution
- 7:42Top 10 List: Why Big Data/Distribution sucks
- 9:49GPU/TPU Services in the Cloud and On-Premise
- 10:19End-to-End ML Pipeline in Python on Hops
- 11:21Google Facets
- 11:53Big Data Preparation with PySpark
- 13:53Model Architecture Search on TensorFlow/Hops
- 14:19Differential Evolution in Tensorboard (4/4)
- 15:16Model Serving: Monitor + Retrain
- 15:48Summary
related talks
34mDeep Learning at Scale
49mDistributed Deep Learning
41mDeep Learning to Solve Challenging Problems
20mSustainability of AI
20mAccelerating AI pipelines w/ NVIDIA on AWS: An autonomous driving case
38mReal-world Machine Learning with TensorFlow and Cloud AI
26mThe scaling laws of AI Why neural networks continue to grow
40mFueling the AI Revolution with Gaming