Use AI for "real things" in your Windows Apps
summary
A working Windows note app adds images, audio and video with on-device OCR using the Windows text recognizer, local transcription via Whisper small.onnx converted with Olive conversion, and ONNX Runtime inference on GPU via DirectML. It indexes content as 384-d embeddings and uses cosine similarity for semantic search, then feeds retrieved passages to a small local language model using ONNX Runtime Generative AI for summarization, autocomplete and RAG-style answers. Demos run offline, show token-by-token streaming, fine-tuning via the Toolkit for VS Code, and integrations like Recall and Windows Studio Effects.
AI-written summary · about
chapters (10)
- 0:00AI in Windows apps
- 3:10AI note-taking app
- 5:40Run AI models locally
- 7:32OCR API demo
- 11:44Transcribe audio with Whisper
- 18:13Semantic search with embeddings
- 24:19Small LMs for summarization
- 30:27RAG for generation
- 34:39Fine-tuning models
- 37:28Responsible AI & closing
related talks
01h 10mPractical Applications of Generative AI: How to Sprinkle a Little AI in Your App
55mDive into the Future: .NET and AI!
59mHow to add genuinely useful AI to your webapp (not just chatbots)
44mAI for every developer
50mCognitive Services Extravaganza!
50mAzure AI, Power new possibilities for every organization
01h 10mPractical Applications of Generative AI: How to Sprinkle a Little AI in Your App
23mFoundry Local: Cutting-Edge AI experiences on device with ONNX Runtime/Olive