Productionizing On-device AI Models
summary
Built an on‑device cutout tool for Instagram called Squeeze segment anything by compressing a server Segment Anything model with pruning, quantization and knowledge distillation, deployed via frameworks like PyTorch Mobile and TensorFlow Lite. Preprocessing uses saliency models; postprocessing uses hole‑filling and edge smoothing. Evaluation reports IoU against the server baseline and manual review showing near‑server quality for this specific task. Claims: radically lower latency, zero server GPU cost and stronger privacy, at the expense of some accuracy.
AI-written summary · about
chapters (17)
- 0:00Intro
- 0:19Ondevice AI
- 4:05Pros and Cons
- 6:30Overview
- 6:48Product Development Cycle
- 7:01ML Readiness
- 7:55Model Compression Techniques
- 9:24PrePostprocessing
- 9:52Preprocessing Techniques
- 11:12Postprocessing Techniques
- 12:14Evaluation
- 13:19Real World Example
- 14:51Three Buckets
- 15:58Gut Check
- 16:10Age of Gen
- 16:22Preprocessing
- 17:36Whats Next
related talks
01h 20mWorkshop1: AI for Everyone
19mOn-device machine learning: TensorFlow on Android
37mHow to Optimize, Validate and Deploy ML Models On Device
46mFire Up Your Smartphone Neurons with On-Device Deep Learning
31mBringing AI on-Device: From Cloud to Edge
39mCreating smaller, faster, production-worthy mobile machine learning models for Android
36mApplied AI on the edge
20mAI and Machine Learning for developers