The Disruptive Potential of On-Device LLMs
summary
Claims that on-device models cut privacy risk and carbon cost and that Llama 3.1 8B and a 70B Llama behave very differently in practice. Demos run a one-line opener and implement Minesweeper: GPT‑4o cloud returns runnable code immediately, code Q 7B (Alibaba) produces runnable code after <5 minutes of fixes, Llama 8B needs 15–20 minutes to correct. Reported metrics: Llama 8B ~73, Llama 70B ~86, GPT‑4o ~86–88, plus emissions figures and a 30B on-device MVP hardware target.
AI-written summary · about
chapters (9)
- 0:00Introduction and Speaker Background
- 1:31Why On-Device LLMs Matter
- 2:15Problems with Large Language Models
- 3:58Benefits of On-Device LLMs
- 4:36State of the Art LLMs: Demos and Comparisons
- 12:13Real World Use Cases of On-Device LLMs
- 13:23Apple Intelligence: On-Device LLM Features
- 17:30Path to MVP: Future of On-Device LLMs
- 20:00Conclusion and Future Applications
related talks
35mOn‑Device LLMs on Android Devices 2025
23mLLMs on the Edge: The Future of On-Device Intelligence
12mKeynote: The Llama Ecosystem: Past, Present and Future
27mProductionizing Open Source LLMs
29mLeveraging open-source LLMs for production
36mBuilding & Deploying LLM Apps
53mLLM Quality Optimization Bootcamp
01h 00mPANEL - Data, Code, and Life with LLMs.