How Observability-First Development Lets You Ship Agents in Weeks...
summary
Claims that building observability in from day one cuts agent time-to-production from 12 months (retrofitted) to 6 weeks (observability-first) and shows the actual pipeline: instrument agents with OpenTelemetry, send traces to OpenSearch via OpenSearch AgentHealth, write five coded agent evals (scored by hand then automated), gate CI/CD with continuous scoring, use sampling + optional LLM-as-judge, store evals with traces, and run locally via a one‑button Apache‑2.0 install.
AI-written summary · about
chapters (13)
- 0:00Observability-first agent dev
- 1:02Agents to services in OpenSearch
- 2:27Debugging agentic workflows
- 4:30Agent traces and evaluations
- 6:39Instrument agents before features
- 8:03Evaluations in CI/CD
- 10:10Traces with agent evaluations
- 11:14Sampling and cost for observability
- 12:15Debugging agent conversation bugs
- 13:18Agent health tool with OpenTelemetry
- 14:20Catching drift with scoring
- 15:01Agent health setup locally
- 16:28Call to action and resources
related talks
36mObservability from 0 to 100
46mObservability in the real world: Improve systems w/iterative approach
33mWhat Did My Agent Do? Observability and Accountability for AI Agents
29mLicense to Observe - Why observability solutions need agents
48mAPI Observability by Design: Building Systems That Tell Their Own Stories
01h 20mMind the Gap (In your Agent Observability)
35mGaining Observability Into Your CI/CD Pipeline
50mEverything You Need To Know About Agent Observability