AI made easier: How the ONNX Runtime and Olive toolchain will help you Q&A
summary
ONNX Runtime plus Olive produce an end-to-end pipeline that converts Whisper (tiny locally, large in cloud) into a single end-to-end ONNX graph that embeds audio preprocessing, beam search and decoder postprocessing so the model accepts raw audio and outputs a string. Olive runs passes including ONNX conversion and CPU int8 quantization, evaluates latency/accuracy, and packages a ZIP with optimized models, NuGet/Python samples and an Azure EP hybrid inference option; ORT Web via WASM/ WebGPU preview runs in browsers. Model files are extractable.
AI-written summary · about
chapters (15)
- 0:00Session overview
- 1:29ONNX runtime for Windows AI
- 3:05Tooling and hardware EPs
- 5:12NPU and web deployment
- 7:07Olive toolchain benefits
- 9:41Whisper model demo
- 13:10Configuring Olive for Whisper
- 16:15Model optimization process
- 18:53C# app integration
- 22:50Web and Q&A introduction
- 24:51Developer questions
- 28:17Technical deep dive
- 33:51Olive vs ONNX runtime
- 37:56Cross-platform strategy
- 44:09Final questions and resources
related talks
39mBuild your enterprise Generative AI applications with Microsoft Olive
39mBuild your enterprise Generative AI applications with Microsoft Olive
05mONNX Runtime
30mScalable ML acceleration with ONNX Runtime
23mFoundry Local: Cutting-Edge AI experiences on device with ONNX Runtime/Olive
36mApplied AI on the edge
30mThe Evolution of ONNX: Driving Interoperability in AI...
42mAI Everywhere– accelerate your development from PC to cloud