From Signal to PR: Anatomy of a Self-Improving Agent
summary
Arize shows that the actual unlock is traces on a filesystem pulled as files into the repo (sometimes ten megabytes), letting an agent produce a real fix — often a one‑line change that lands as a pull request with the fix waiting for review. The system runs chosen harnesses in sandboxes or inside a VPC so companies like Uber and Booking keep production private. The recommendation: trace ten times more so agents have exact code paths, not guesses.
AI-written summary · about
chapters (12)
- 0:00Arize, its agent Alyx, and why v1 sucked
- 1:36Observability is changing: from dashboards to telemetry for agents
- 2:55The goal: systems that fix themselves
- 4:14Inverting the loop: the agent investigates first
- 6:08Traces on a filesystem, the key unlock
- 7:23From your laptop to sandboxes
- 8:13A real fix: the Alyx stream canceled bug
- 9:39Why you should trace ten times more
- 11:10Product demo: Signal, AX, and Phoenix
- 13:09Sandboxes, VPC, and why customers won't call out
- 16:20Q&A: why not just point Claude Code at your data?
- 18:04Q&A: where do the evals come in?
related talks
16mSelf Driving Products: Product Signals to Pull Requests
30mAgents Building Agents
23mFrom Blind Spots to Merged PRs: Continuous Agentic Performance Optimization
34mFinding Signal in the AI Noise
31mInternal Background Agent Systems: Architecture and Rollout
20mThe Missing Layer After Launch
34mAI Agents for Performance: Ship Faster, Pay Less
12mHow to maintain a codebase – when everyone can code