The Lean LLM development playbook
summary
Wants specifics: a claimed workflow called the GenOp cycle with concrete steps (plan, model, workflows, evals, data, deploy, operate, observe), tools named like LangChain and vector DB causing deployment headaches, evaluation tooling pointers (Martian llm leaderboard, limes chatbot Arena), concrete failure modes (prompts that work on 3.5 break on 4.0, timeouts of two to three minutes), and a built product — deni.com works and is free — plus mitigation patterns (chains, agents, queues, evals) to make it reproducible.
AI-written summary · about
chapters (7)
- 0:00Dinner app to LLM struggles
- 4:43Traditional DevOps fails with LLMs
- 10:22GenOps development cycle
- 16:19Workflow with prompt chains and agents
- 20:07Data integration and system deployment
- 23:58Monitoring LLM applications
- 25:44Key takeaways and best practices
related talks
38mNobody Pages the LLM: Engineering Rigour for Vibe Coding
35mEvaluating LLM systems: the nuts and bolts
19mLLM codegen fails and how to stop 'em
38mMaking LLMs reliable - A practical framework
29mCustomizing and Evaluating LLMs, an Ops Perspective
28mA community based open source LLM and what it can do for your business
43mAI Development Lifecycle: Learnings of What Changed with LLMs
01h 00mFrom LLM Theory to Practical Agentic Implementations