Break It 'Til You Make It: Building the Self-Improving Stack for AI Agents
summary
Claims a deployed agent needs a continuous monitoring and improvement stack that actually connects evals, tracing, observability, experimentation and optimization into a closed loop. Shows a working tracing system that surfaces tool-call correctness and argument-level failures, measures trajectory/order correctness across traces and multi-turn sessions, runs experiments that change prompts and tool wiring, and iterates eval prompts themselves with a golden dataset using Arise Phoenix and Riseex to drive production self‑improvement.
AI-written summary · about
chapters (5)
related talks
30mBuilding Reliable AI Agents: An Open Source Approach To Evaluation and Observability
36mAdvanced Evaluation Techniques for AI Agents
01h 20mMind the Gap (In your Agent Observability)
47mHow to Build Enterprise-Grade AI Agents Using Robust Evaluation
35mThe Agentic AI Engineer
26mBeyond Sandboxes: Architecting Durable Runtimes for AI Agents
27mHow to Build AI Agents and Keep Your Sanity
40mBuilding Effective AI Agents Simple, Composable Patterns