Scalable ML acceleration with ONNX Runtime
summary
ONNX Runtime is an open‑source inference engine that converts TensorFlow/PyTorch models to ONNX and runs them via pluggable Execution Providers (OpenVINO, TensorRT, Arm Compute Library, CPU fallback). Claim: one API lets identical containerized code hit CPU, GPU, NPU or DSP without per‑hardware custom runtimes. Microsoft ships converters, an ONNX Model Zoo, and a Jetson Nano multi‑container GitHub sample built with the Garage team that demonstrates accelerated edge inference running end‑to‑end.
AI-written summary · about
chapters (23)
- 0:00Introduction
- 0:50Why now
- 4:11Intelligent edge
- 4:35Intelligent edge spectrum
- 6:07ML development workflow
- 7:27Edge ML
- 8:03ML Lifecycle
- 11:21ML Challenges
- 12:58What is ONNX
- 14:08Versioning
- 14:52Conversion
- 15:11ONNX Models
- 15:43Converters
- 15:56Deployment
- 16:26ONNX Runtime
- 18:51Supported Architecture
- 19:59AI requirements
- 21:41Deployment options
- 22:32On the edge
- 23:16Multiple container options
- 24:25Single container
- 25:19Multiple containers
- 26:37Microsoft Garage
related talks
05mONNX Runtime
33mWhat on earth is ONNX and ONNXRUNTIME
45mLeverage Power of Machine Learning with ONNX
41mLeverage Power of Machine Learning with ONNX
30mThe Evolution of ONNX: Driving Interoperability in AI...
42mAI Everywhere– accelerate your development from PC to cloud
37mDeep Learning Neural Network Acceleration at the Edge
51mMaking neural networks run in browser with ONNX