The Extract/Validate/Extrapolate Loop
summary
Introduces the extract/validate/extrapolate loop as a concrete UI and engineering pattern: use an LLM to extract structured facts, run a validation step (human, formal checker, or machine using token log probabilities and unit-test comparisons), then drive actions or state transitions (examples: CoPilot suggestions, Sublayer running unit tests, Stately state machines, DSP prompt-tuning). Decomposes monolithic prompts into composable blocks, reduces hallucinations, makes outputs debuggable, and is already used in a mentoring beta alongside human coaches while standalone workflows are being iterated and measured.
AI-written summary · about
chapters (9)
- 0:00Introduction and setup
- 1:04Audience questions on AI models
- 3:09Motivating problem for LLM applications
- 6:38Introducing extract validate extrapolate Loop
- 12:40Designing reliable LLM applications
- 17:15Q&A on mentoring applications
- 20:41Machine validation approaches
- 23:54Asynchronous validation challenges
- 24:57Feedback loops and user knowledge
related talks
22mStop Prompting, Start Looping: Principles of Loop Engineering
52mBuild a Prompt Learning Loop
50mFrom Prompt Hacking to Architectural Determinism: Engineering Reliable GenAI Systems
39mThe Boringly Simple Loop Powering GenAI Apps
27mThe Lean LLM development playbook
53mSemantics-based program manipulation in the age of LLMs
29mYour Prompt Is Not a Unit Test — instrumenting LLM apps in production
43mAI Development Lifecycle: Learnings of What Changed with LLMs