Developer Productivity for GenAI Apps
summary
Argues that production GenAI needs standard dev workflows: local small models (ollama, llama.cpp, TGI) run in containers for parity, CI evaluation using the Ragas framework, and fine‑tuning via Instruct Lab for stable domain knowledge. Recommends treating models as software—use separate LLMs to fact‑check outputs and to scan prompts for injection, externalize prompts and enforce structured output schemas and few‑shot examples, and run statistical evals in CI to catch regressions rather than relying on manual checks.
AI-written summary · about
chapters (21)
- 0:00Introduction to talk
- 1:07What is AI today
- 3:12Core pillars of productivity
- 6:01AI commoditization and roles
- 8:10AI becoming part of apps
- 10:18DevOps with AI components
- 11:00Handling model non-determinism
- 12:52Fact-checking AI outputs
- 14:15Testing AI components
- 15:18Security for AI inputs
- 17:06AI setup complexity
- 17:48Prompt engineering basics
- 19:57Externalizing AI prompts
- 21:00Evaluating AI models
- 23:10Fine-tuning models
- 24:58Infrastructure for AI
- 26:21Local AI model execution
- 27:25Small models for specific tasks
- 28:29Containers for AI models
- 29:12Application architecture with AI
- 30:14Summary and predictions
related talks
20mDeveloper Productivity in the Age of GenAI
15mChallenges in moving Gen AI apps to production
33mOperationalizing GenAI Applications with DevOps Practices
41mGenerative AI: bliss or miss?
56mPANEL: Unleashing the Future: The Impact of Gen AI on Development Productivity
47mEmbedding AI: Don’t Burn Your Fingers in the Heat of the AI revolution
36mUnleashing Generative AI: Improving Developer Productivity in SRE
19mDeveloper Centric AI