30 Golden Rules of Deep Learning Performance
summary
Thirty concrete optimizations target GPU starvation: rewrite thousands of tiny images into TFRecord ~100MB shards, use TFDS + prefetch + num_parallel_calls ≈ CPU cores, cache and interleave IO, and push augmentations to GPU with NVIDIA DALI -> ImageNet in 80s. Training gains include automatic mixed precision 2–3×, tune batch sizes to hardware (multiples of 8/64/256), use LR finder, autotune, and distributed strategies; for inference apply quantization/pruning (InceptionV3 95MB→23MB, 78%→77% top1) and enable GPU persistence.
AI-written summary · about
chapters (20)
- 0:00Introduction and context
- 3:17Transfer learning explained
- 6:31GPU starvation and optimization goals
- 8:19Profiling tools for performance
- 13:13Data preparation rules
- 18:57Data reading optimizations
- 27:11Data augmentation on GPU
- 30:24Automatic mixed precision
- 36:38Batch size and hardware tuning
- 41:44Learning rate optimization
- 44:18Eager execution vs graph mode
- 45:43Overtraining and generalization
- 49:15Optimized software stack
- 53:12Parallel CPU threads
- 54:41Multi-GPU training
- 57:09Industry benchmarks
- 1:01:46Inference optimization
- 1:06:53Model pruning and quantization
- 1:09:27Fused operations
- 1:10:10GPU persistence
related talks
30mUnderstanding GPUs
08mHacks to Make LLM Training Faster
28m🧼 From GPU-poor to data-rich: data quality practices for LLM fine-tuning
26mDeepSpeed: Efficient Training Scalability for Deep Learning
12mThe Ultra-Scale Talk: Scaling Training to Thousands of GPUs
15mDesign Compact Deep Learning Models: Small is the New Big
37mAn Initial Look at Deep Learning IO Performance
01h 18mOptimizing LLM Training and Inference Performance on GPUs