Escaping the Cloud: High-Performance AI in your Browser
summary
Argues that full inference can move into the browser using WebGPU, ONNX Runtime and Transformers.js, and demonstrates LLMs, speech and vision running locally (pose estimation at ~10 FPS on an Intel integrated GPU, also running on a Fairphone 5). Shows concrete tactics: quantization to int8/4-bit, WASM builds, and WebNN/WindowsML/CoreML fallbacks. Notes hard limits: ~3 GB downloads for a 4B model, browser cache eviction and ~4 GB WASM memory ceilings, and uneven WebNN support across browsers in 2026.
AI-written summary · about
chapters (10)
- 0:00Introduction to web AI
- 3:41Why run AI locally
- 8:42Hardware for local AI
- 12:57Tools for browser AI
- 18:58Frameworks for AI in browser
- 23:37Optimizing AI models
- 26:48Challenges of web AI
- 30:22Live demos and examples
- 34:37Future of web AI
- 36:04Q&A session
related talks
46mThe Ultimate Guide to Browser-Based AI: Privacy-First, Offline-Ready and Future-Proof
17mRunning Intelligence at the Edge: Web AI in Action
45mPrivacy-First In-Browser Generative AI Web Apps
33mLocal Models: How to Run AI in the Browser
46mFrom ML to LLM: on-device AI in the browser
32mPrivacy-first in-browser Generative AI web apps
36mApplied AI on the edge
39mThe AI PC and the Era of the Modern Web