Architecting AI: Efficient Models, Everywhere
summary
Arm argues that small models let AI run on device to cut token fees, lower latency and keep private data local, citing a 300 ms UX ceiling and that training frontier models uses the power of a small city. Concrete tools and claims: ExeutorTorch 1.0 for export/optimization, ARM Lumx and Clyde for heterogeneous inference, fine‑tuning approaches like adapters, RAG and PEFT, and examples from 80B Qwen down to a 7 million‑parameter model that excels on specific tasks.
AI-written summary · about
chapters (9)
- 0:00Efficient AI everywhere
- 1:07Small AI models game changer
- 3:13Large vs small model trade-offs
- 5:37Agentic workflows token economics
- 10:30Local inference faster UX privacy
- 13:40On-device AI performance
- 15:04Fine-tuning small LMs
- 16:31ARM tools for AI deployment
- 18:18Summary call to action
related talks
36mApplied AI on the edge
20mFireworks AI: Go from Hype to High - Value AI
46mSmall AI is the next Big Thing
55mEdgeAI - AI in the Palm of Your Hand
16mEdge AI needs to be for everyone: breaking barriers to adoption
48mThe Power of Edge AI: Revolutionizing Future of Real-Time, Autonomous Processing
15mDesign Compact Deep Learning Models: Small is the New Big
14mArm: One Platform for Agentic AI, from Cloud to the Edge