Deliver AI-powered experiences across cloud and edge with Windows
summary
Demonstrates taking Whisper and Stable Diffusion through Olive to produce packaged, hardware-optimized ONNX models with execution providers for CPU, GPU, NPU and cloud. Olive yields >2x speed and <50% model size for Whisper and delivers Stable Diffusion speedups from 8s to 1.3s on an RTX 4090 (≈6x) and from 10s to 2.8s on an RX 7900 (≈3.5x). Hybrid routing switches between Azure Execution Provider and local EPs by runtime conditions, deploys to browser, Android, Windows, and ships with ONNX Runtime binaries and sample code. DirectML EP provides cross‑GPU Windows acceleration.
AI-written summary · about
chapters (7)
- 0:00Introduction to hybrid AI
- 1:28Hybrid loop and Olive toolchain
- 6:35Hybrid AI demo showcase
- 11:44Whisper model optimization
- 22:24DirectML execution provider
- 28:30End-to-end performance demo
- 35:37Recap and developer resources
related talks
39mBuild your enterprise Generative AI applications with Microsoft Olive
39mBuild your enterprise Generative AI applications with Microsoft Olive
23mFoundry Local: Cutting-Edge AI experiences on device with ONNX Runtime/Olive
38mBring AI experiences to all your Windows Devices
36mApplied AI on the edge
42mAI Everywhere– accelerate your development from PC to cloud
05mAccelerate AI development from cloud to client
18mWeb AI on next generation AI PCs