Lightning Talk: High Performance Neural Network Inferencing Using WASI NN
Lightningat Cloud Native Wasm Day North America 2021 · on Tuesday, 2021-10-12· at 16:05· Room 501 ABC + Online·
summary
WASI-NN exposes four simple calls to load model bytes, create an execution context, bind tensor inputs and run compute so runtimes can perform near-native inference with hardware acceleration. Implementations exist for OpenVINO and ONNX and run the same WebAssembly module across Raspberry Pi, Intel/AMD CPUs and GPUs via Wasmtime. Reported numbers show model inference around 2 ms while tensor preprocessing is roughly 10× slower, and workflow supports PyTorch and TensorFlow models.
AI-written summary · about
chapters (4)
- 0:00Start
- 0:591. WebAssembly and WASI 101
- 4:18OpenVINO
- 6:311. the actual inference is really fast (ams)
related talks
22mMachine Learning with Wasm (wasi-nn)
27mAI inference on the Edge cloud using WebAssembly
07mLightning Talk: AI at the Edge: ONNX Inference in WASM on Featherweight K0s
33mBringing AI to the Heterogeneous Edge with WebAssembly & ONNX
37mGetting Started with AI and WebAssembly
16mThe Web is AI Ready—maximize your AI web development with WebNN
32mWebAssembly Based AI as a Service on the Edge with Kubernetes
08mLightning Talk: Accelerate WebAssembly AI with Backend Graph Compliers on a Wide Variety of AI HW Accelerators