Bringing AI on-Device: From Cloud to Edge
summary
Compares cloud inference versus on-device tradeoffs and shows working examples: DETR quant at ~19 FPS on an iPhone 13 (sped 1.5×), Stable Diffusion 1.5 running on an older Android taking ~1.5 minutes per image, and practical stack choices like Core ML, TensorFlow Lite, MediaPipe and WebAssembly runtimes. Cites concrete device wins — Apple Face ID’s 30,000-dot scan and Apple Watch AFib detection (≈99.6% in a ~600-person trial) — and recommends resources (Harvard ML Systems, TinyML) and GitHub demos for replication.
AI-written summary · about
chapters (11)
- 0:00Introduction
- 2:47Why AI on device matters
- 8:13Real-world AI on-device use cases
- 11:24Cloud AI inference explained
- 14:15Advantages of on-device AI
- 15:39Challenges of edge AI
- 17:47WebAssembly for edge AI
- 20:17Core ML for Apple
- 23:57Google AI Edge framework
- 26:08Key resources
- 28:02Q&A
related talks
23m2025: On‑Device AI Goes Mainstream on Android
55mEdgeAI - AI in the Palm of Your Hand
42mOn-Device Machine Learning with Google AI Edge
32mAI for Mobile and IoT Devices: TensorFlow Lite
01h 20mWorkshop1: AI for Everyone
01mWhy edge AI is becoming the default for devices
15mSimplified Machine Learning on mobile, browsers, and edge devices with on-device ML
43mMyths and Reality of Android Machine Learning