Boring but Effective: WebAI That Ships
summary
Demonstration of browser-first ML with Transformers.js shows a working kitchen assistant that runs fully offline but requires a 3 GB download for all models; real-world numbers: Gemma model hits ~140 tokens/s on an M3 and only ~3 tokens/s on an older Dell. Concrete demos: Whisper tiny (≈100 MB) transcribes 3–4s of audio in 0.27s, a 26 MB depth model runs ~34 FPS, and 1,020 text embeddings compute in 3s with similarity queries at ~17 ms. WebGPU kernels and cross-origin storage are the practical next steps.
AI-written summary · about
chapters (10)
- 0:00On-device AI intro
- 3:29Hidden challenges of on-device AI
- 7:43Optimizing AI runtimes for browser
- 8:05Pipeline API for web AI tasks
- 10:28Real-time audio/image tasks
- 12:18Text embeddings and similarity search
- 15:07Why run AI on device
- 16:30Client vs server AI decisions
- 19:38Privacy-focused medical app
- 21:42Conclusion and future
related talks
25mTransformers.js: Building Next-Generation WebAI Applications
46mFrom ML to LLM: on-device AI in the browser
20mFrom ML to LLM: On-Device AI in the Browser
30mLook ma, no hands! Multimodal AI-Agents in the browser
41mEscaping the Cloud: High-Performance AI in your Browser
24mWeb Apps of the Future With Web AI
29mClient side Web AI Agents Building smarter user experiences for a future age
33mLocal Models: How to Run AI in the Browser