React + WebLLM: Building Intelligent Applications with In-Browser Language Models
summary
Shows how to run LLMs directly in a React app using the MLC/WebLLM stack with WebGPU, loading a 1B Llama tiny (~1–2GB) model in the browser, caching it client-side and serving chat completions with optional streaming. Demo uses the createMLCEngine API, supports multiple models (Llama, Gamma, F3 mini, Microsoft), deploys on Vercel with no server compute, and reports reduced latency 2–3s versus remote calls while keeping models cached client-side for subsequent fast starts.
AI-written summary · about
chapters (7)
- 0:00Introduction and motivation
- 2:30What is WebLLM
- 7:05How WebLLM works under the hood
- 8:12Setting up WebLLM in React
- 12:07Live demo of WebLLM
- 14:16Deployment and performance benefits
- 15:40Future possibilities and conclusion
related talks
30mP2P Distributed LLM Inference Across Browsers With llama.cpp And WebRTC
31mFrom ML to LLM on device AI in the browser
34mPrivacy First: Building LLM-Powered Web Apps with client side WASM
40mOpen Source LLMs in the Cloud: Scalable Solutions
46mFrom ML to LLM: on-device AI in the browser
27mBuilding Useful Apps With Small, Local LLMs
17mRunning WebLLM in the Browser on RISC-V Toward Lightweight, Local AI Experiences
45mEverything You Need to Know About Running LLMs Locally