AI inference on the Edge cloud using WebAssembly
summary
Compares approaches to run TensorFlow inference via WebAssembly with concrete timings: interpreted Wasm is slow, JITed Wasm (TensorFlow.js style) gives 300–500x speedup to a few seconds, ahead-of-time compiled models cut that to ~1–2s, calling native TensorFlow from Wasm via a Rust SDK hits ~0.5s, and running native libraries on AI chips/GPU reaches ~0.05s (~20 fps). Presents tools and projects: TensorFlow, ONNX, SSVM/WasmEdge, Qualcomm integrations and WASI-like TensorFlow extensions, plus code and demos.
AI-written summary · about
chapters (11)
- 0:00Start
- 2:20Performance challenges in Al inference
- 4:00Why not Python?
- 9:03What about native apps
- 12:15WebAssembly (Wasm) advantages
- 15:21Interpreted on CPU
- 16:25Just-In-Time (JIT)
- 17:21Compiled from models
- 18:53Host functions (WASI)
- 20:17AOT and GPU
- 24:49What's next
related talks
33mWebAssembly based AI as a Service with Kubernetes
41mWebAssembly Based AI as a Service
37mAI/ML at the Extreme Edge with WebAssembly: A Path Forward
32mWebAssembly Based AI as a Service on the Edge with Kubernetes
26mPaving the Way of WebAssembly in the Intersection of Machine Learning and Cloud Native
37mGetting Started with AI and WebAssembly
01h 04mDevelop, Build & Deploy Lightweight AI Services with WebAssembly
40mWebAssembly for AI Infra : A Lightweight, Fast, and Secure Alternative Approach