Client Side Deep Learning Optimization with PyTorch
summary
Shows how PyTorch models get off the server and run on phones: serialize with torch.jit.script into .pt (libtorch C++), call model.eval, warm up and cache, and strip unused ops via an exported YAML plus static linking. Covers quantization methods—dynamic quantization, static quantization and quantize‑aware training—with numbers: MobileNet drops from 22 MB and 0.13 s to 5.6 MB and 0.03 s with ~74%→73% accuracy; an LSTM goes 169→134 MB and 5.0→1.6 s. Also module fusion and channel last format.
AI-written summary · about
chapters (22)
- 0:00Intro
- 0:26Who are we?
- 2:57Why PyTorch?
- 5:18Advantages of Eager Execution
- 5:48Optimization necessitates looking under the hood
- 6:31Axes of Optimization
- 10:16Production Considerations
- 12:55Scripting Handes control flow and other arbitrary
- 13:33Scripting + Tracing
- 14:31Intermediate Representations in Pytorch
- 16:20Running in C++
- 16:57Speed tips
- 18:43Running Arbitrary Models
- 20:18Lite Interpreter
- 20:48What is Quantization?
- 21:18Quantization in PyTorch
- 22:03Eager Mode Quantization
- 22:50Dynamk Quantization
- 27:19Quantized Aware Training
- 28:48Experimental Results
- 31:16Channel Last Format
- 32:09Addendum
related talks
46mFire Up Your Smartphone Neurons with On-Device Deep Learning
10mQuantization
08mPyTorch Mobile
39mCreating smaller, faster, production-worthy mobile machine learning models for Android
14mOptimize your TensorFlow Lite models
17mDeploy a custom Machine Learning model to mobile
29mRunning Deep Learning Models on Mobile with optimised speed
18mDeploying ML solutions with low latency in Python