Small AI is the next Big Thing
summary
Argues that tiny models (Gemma 4, Gemma 3, Tiny Llama, Quen) are now practical: fine-tune with LoRA on ~1,000 examples in ~2 hours on Colab, quantize to 4‑bit (or experimental 1.5–2‑bit), convert to FlatBuffer/LightRT and deploy. Gives concrete wins: a Sub‑Saharan team moves drug‑authenticity checks on‑device to remove cloud latency, ARM’s Scalable Matrix Extensions speed Gemma3 chat ~6×, and on‑device summarization reaches ~800 words/second. Bench and battery test before release.
AI-written summary · about
chapters (11)
- 0:00Introduction
- 4:12Large AI limits and small AI rise
- 8:26Why small models work
- 13:24On-device AI advantages
- 17:14Skills for small AI developers
- 22:36Android complexity for on-device AI
- 27:49Agentic workflows and user intent
- 32:21Multilingual and domain-specific AI
- 35:49Small AI limits and deployment checklist
- 38:59Future of small AI on edge
- 39:20Q&A
related talks
24mAccelerating AI on Edge
23m2025: On‑Device AI Goes Mainstream on Android
21mFrom 46% to 90%: Fine-Tuning Tiny LLMs for On-Device Agents
41mBuild intelligent Android apps with Google's AI
16mGemma 4: Compacting Intelligence for the Edge
22mBoring but Effective: WebAI That Ships
20mArchitecting AI: Efficient Models, Everywhere
19mReal-Time AI: Low Latency Solutions for Interactive Applications