Zero to Hero - Develop your first app with Local LLMs on Windows
summary
Walks through a working pipeline that fine tunes Phi-3-mini (3.8B params) with Olive and QLoRA on PyTorch inside WSL, installs dependencies with WinGet, and produces a local GitHub issue labeler that applies correct triage labels in seconds and never leaves the box. Shows a second app that indexes a 1,300 page PDF into embeddings with a 90 MB ONNX model and uses RAG to return grounded answers in about 10–15 seconds for indexing and fast inference.
AI-written summary · about
chapters (8)
- 0:00Introduction to local AI
- 2:07App 1: GitHub issue triage
- 5:54Fine-tuning walkthrough
- 8:57Demo: Local issue triage
- 18:02App 2: PDF query tool
- 23:34Setting up the environment
- 29:45Demo: PDF RAG application
- 37:54Conclusion and resources
related talks
43mLarge Language Models for Devs: from zero to your first LLM application
47mOwn Your LLM: Running Generative and Agentic AI Locally
37mEmpowering Developers with local AI
43mSupercharge Your PowerShell - Harnessing...
38mQuerying Live Data With LLM App
27mBuilding Useful Apps With Small, Local LLMs
01h 19mBuilding an LLM on your Laptop to...
30mLearning to wield large language models