High Performance Deep Learning on Edge Devices With

on YouTube ↗33m

" summary

Apache MXNet ships a mobile runtime under 10 MB and shows a full vision network running near real time on a Raspberry Pi, using optimizations like separable convolutions and quantized weights and deploying models from a model zoo. Training scales linearly to 256 GPUs and inference benefits from 80% sparsity with little accuracy loss. Greengrass enables hub‑and‑spoke local learning where Raspberry Pis send captures to a Jetson that refines and pushes improved models back, reducing cloud bandwidth and latency.

AI-written summary · about

" chapters (24)

1. 0:00Intro
2. 3:21Motivating Examples
3. 5:49Peep Learning Frameworks
4. 6:38Apache MXNet | Flexible Programming
5. 7:53Apache MXNet | On Mobile Devices
6. 10:55Apache MXNet | Easy to Get Started
7. 12:00The Metal: Heterogeneity
8. 12:53The Metal: Performance Gap
9. 14:27The Models: Where is Our Cost?
10. 15:46Cheaper Convolutions: Winograd
11. 16:51Cheaper Convolutions: Separable Convolutions
12. 18:14Depth Separable Convolutions in MS
13. 18:56Fewer Parameters: Quantization
14. 20:25Fewer Parameters: Weight Pruning Prune unused weights during training
15. 21:09Weight Pruning in MXNet
16. 22:52Fewer Parameters: Tensor Decompositions
17. 23:55Table of Model Optimization Techniques
18. 24:46Edge Model Optimization Benefits The Cloud
19. 26:13The Challenge For Artificial Intelligence: SCALE
20. 27:17AWS Tools for Deep Learning
21. 28:12AWS Deep Learning AMI: One-Click Deep Learning
22. 29:25AWS IoT and AWS Greengrass
23. 30:01Manage and Monitor Models on The Fly
24. 31:05Local Learning Loop

" related talks

• 37mDeep Learning Neural Network Acceleration at the EdgeAndrea Gallo · Open Source Summit Europe & ELC + OpenIoT Summit Europe 2018
• 49mEnable IoT with Edge Computing and Machine LearningJared Rhodes · Update Conference Prague 2018
• 15mDesign Compact Deep Learning Models: Small is the New BigDavis Sawyer · GOTO Chicago 2020
• 27mModel Serving at the Edge Made EasierAnimesh Singh, Paul Van Eck · Kubernetes on Edge Day Europe 2022
• 55mUsing TensorFlow for Performing Deep Learning at the EdgeAditya Bhattacharya · Developers Summit 2021
• 48mThe Power of Edge AI: Revolutionizing Future of Real-Time, Autonomous ProcessingDeepak Das · DevConf.IN 2026
• 28mMachine Learning on the EdgeFrank Kelly · PyCon UK 2019
• 20mApache MXNet, a highly memory efficient deep learning frameworkGirish Patil · Anthill Inside 2017