Lessons Learned Building AI-powered Features in VS
summary
Built and shipped an AI-powered rename suggestions feature integrated via GitHub Copilot, used by ~40,000 people, that sends a focused code snippet to an LLM and expects a JSON with three name choices. Work centers on prompt engineering (few‑shot examples, RAG when needed), handling token limits and non‑deterministic outputs, measuring quality by acceptance rate and error rate, and controlling COGS by invocation design (button → toggle + delay), sending minimal context, and choosing smaller or older models.
AI-written summary · about
chapters (14)
- 0:00Intro
- 1:07About Dalia
- 5:00Learn how AI works
- 8:08Strengths
- 10:23Weaknesses
- 16:30Build Something
- 19:46Red Flags
- 25:26Simple Techniques
- 31:53Prompt Engineering
- 37:29Try Different Things
- 41:07Use Data
- 45:21Responsible AI
- 49:33Cogs
- 56:14Enjoy the Journey
related talks
36mME, MYSELF AND AI
46mAI Integration for Devs: ChatGPT and Copilot in Your Projects
53mImprove Your Code Quality and Performance using AI and GitHub Copilot
01h 00mLevel up with GitHub Copilot: using AI to learn, code, and build
48mMastering the AI Shift: Adaptive Strategies for Software Evolution
25mImproving developer happiness with AI
39mPower of AI and Code Quality and Performance
01h 01mUnlocking the Potential of AI