High Performance Distributed Tensorflow
summary
Live demos apply 8-bit quantization, weight‑rounding and batch‑normalization folding to trained TensorFlow graphs, then loadtest optimized vs unoptimized models to show concrete tradeoffs: model files shrink (example shown from ~80 MB with runtime to ~500–600 KB with TF‑compile AOT), throughput rises with request batching on GPUs while single‑request latency can increase, and the Tensorflow Graph Transform Tool is used to strip nodes and validate that the optimized binary actually serves correct predictions.
AI-written summary · about
chapters (15)
- 0:00Introduction
- 7:19Optimizing TensorFlow for production
- 11:58Understanding TensorFlow models
- 12:41Distributed TensorFlow evolution
- 14:08Data pipelines and queues
- 15:54XLA and graph optimizations
- 18:46Post-training model optimizations
- 21:56AOT compilation and graph simplification
- 22:59TensorFlow Serving optimizations
- 25:07TensorFlow with GPUs and half precision
- 34:09CUDA streams and integration
- 35:55Model deployment and prediction
- 37:42Distributed TensorFlow architecture
- 38:46ExcelA prediction optimizations
- 40:34Q&A
related talks
36mHigh-Performance Distributed Tensorflow
14mOptimize your TensorFlow Lite models
59mBuilding models for TPU devices
34mTensorFlow Lite
21mThings I learned running neural networks on microcontrollers
43mTensorFlow for Mobile Developers
25mThe Golden Triangle of Inference Optimization: Balancing Latency, Throughput, and Quality
41mTensorFlow ever higher