Video AI in Your Browser with WebGPU
summary
Browser-based video captioning pipeline that actually runs on the GPU: use the File API to load a video, draw frames to canvas, assemble a three-frame filmstrip, and run a vision‑language model via Transformers.js on WebGPU runs AI in‑browser. Demonstrated model is Apple fastBLM (about fastBLM 300MB), the live demo returns a sensible caption. Performance gains reported: 20 fps with WebAssembly, 200 fps with WebGPU and >1000 fps with WebN on NPU hardware, with the tradeoff of larger model downloads and lower accuracy for tiny models.
AI-written summary · about
chapters (13)
- 0:00AI and WebGPU intro
- 3:11Web evolution and WebAssembly
- 5:41What is WebGPU
- 7:49AI on GPUs with WebGPU
- 8:31Video captioning app
- 10:35Transformers.js for AI
- 12:40Vision language models
- 14:45Live demo: video captioning
- 15:48Future with WebNN
- 17:55Performance gains
- 18:39Limitations and trade-offs
- 19:42AI in web predictions
- 20:44Q&A
related talks
29mImage Processing in the Browser
22mBoring but Effective: WebAI That Ships
41mEscaping the Cloud: High-Performance AI in your Browser
31mNode.js <=> Machine Learning
45mPrivacy-First In-Browser Generative AI Web Apps
25mTransformers.js: Building Next-Generation WebAI Applications
24mThe next level apps for Web developers Machine learning on browsers 🤯
41mAI in Web: TensorFlow js