Things I learned running neural networks on microcontrollers
summary
Shows concrete decisions: deploy on edge for very low latency, privacy, poor networks, or high throughput; use federated training if data must never leave devices. Uses TensorFlow Lite model optimization toolkit and compares methods: post-training float16 quantization gives about 3–4× size reduction with negligible accuracy loss, while dynamic range or 8-bit gives ~5× shrink with higher error. Also evaluates weight sharing (≈50% cut) and pruning; reports which hardware supports which trick.
AI-written summary · about
chapters (9)
- 0:00Edge neural networks intro
- 1:12Deploy models on edge
- 4:20Training vs inferencing at edge
- 7:06Inferencing on microcontrollers
- 8:30Reduce model size with quantization
- 11:20Choosing quantization methods
- 15:38Weight sharing and pruning
- 17:24Benchmarking edge neural networks
- 20:15Conclusion
related talks
14mOptimize your TensorFlow Lite models
59mBuilding models for TPU devices
01h 01mML and the IoT: Living on the Edge
39mCreating smaller, faster, production-worthy mobile machine learning models for Android
27mMachine Learning on the edge using TensorFlow Lite
36mHigh-Performance Distributed Tensorflow
49mMobiconf 2018: Vladimir Alekseichenko, DataWorkshop
34mTensorFlow Lite