Build a Prompt Learning Loop
summary
Builds a practical prompt optimization loop that reads failure examples, asks an LLM to rewrite the system prompt using human explanations and rule-checker feedback, then evaluates with Elements as judge and OpenAI models; a coding-agent case improves accuracy by ~15% (4.1 toward 4.5) at about two thirds the cost. The notebook runs async JSON webpage experiments with train/test split, controllable loops and temperature 0, and claims better results than the evolutionary GA baseline in fewer iterations. repo and notebook provided uses rule-by-rule evaluators
AI-written summary · about
chapters (9)
- 0:00Introduction
- 1:59Agents fail in production
- 4:50Prompt learning basics
- 9:01Case study: effectiveness
- 13:38Prompt learning vs GA
- 23:05Setting up the prompt learning workshop
- 36:55JSON webpage prompt
- 43:13Running optimization loop
- 51:31Q&A
related talks
52mBuild a Prompt Learning Loop
11mThe Unreasonable Effectiveness of Prompt Learning
30mIs Prompt Engineering Dead? How Auto-Optimization is Changing the Game
01h 39mMastering LLMs: From Prompt Engineering to Agentic AI
14mPrompt Engineering is Dead
26mMastering Large Language Models: Prompt Engineering for Precision & Performance
49mPrompt Darwinism: Evolutionary Engineering in the Age of LLMs
39mOptimizing a Prompt for Production