Bringing and Running AI Agents in the Browser
summary
Covers running LLM inference fully in-browser using WebAssembly runtimes, WebGPU acceleration and transformers.js / MLC WebLLM, including quantization to fit models under the ~4GB wasm limit and a live demo loading a 2.5GB Gemma model in the page. Explains function-calling via structured JSON tool calls, on-device RAG with embeddings and vector search, runtime choices (webgpu vs CPU wasm), and tradeoffs: privacy, cost savings, and when browser agents outperform cloud-hosted architectures.
AI-written summary · about
chapters (11)
- 0:00AI agents in browser
- 5:31Cloud AI limits
- 7:39JavaScript vs AI workloads
- 10:06WebAssembly & WebGPU for AI
- 13:17How LLMs work
- 15:45AI agents with function calls
- 18:38Retrieval-Augmented Generation in the browser
- 22:31AI agent design patterns
- 25:18JS frameworks for AI agents
- 27:03Future of AI in constrained envs
- 28:07Live demo: WebLLM
related talks
33mLocal Models: How to Run AI in the Browser
22mClient side Web AI Agents: Building smarter user experiences for future agentic ...
29mClient side Web AI Agents Building smarter user experiences for a future age
01mJavaScipt & AI
02mDiscover Web AI: Client side Agents, Gen AI, and machine learning in the browser
21mUnlocking Client-Side AI in Your Web App
24mState of client side AI
15mThe future of machine learning using JavaScript