Anthropic Workshop: Build Agents That Run for Hours
summary
Argues that concrete harness patterns beat vague promises: use an adversarial evaluator (separate LLM that runs Playwright and grades by rubric) paired with a generator, decompose work via a Planner and negotiate generator‑evaluator contracts, and store persistent artifacts on disk. Gives numbers and artifacts: model progress from ~1 hour (Sonnet/Opus 3.x) to ~12 hours (Opus 4.6), many generated apps run 3–5 hours, an example Retro Forge cost ≈$200 over 6 hours, and 27 contract criteria fixed bugs; reading traces is the primary debugging loop.
AI-written summary · about
chapters (20)
- 0:00Introduction and speakers
- 1:21Overview of long-running agents
- 2:29Challenges: Context, Planning, and Judgment
- 4:14Two approaches: Model updates vs. Harness evolution
- 5:58Prehistory: Sonnet 3.5, Computer Use, and MCP
- 6:34The evolution of Claude Code
- 7:55The Ralph loop technique
- 9:49Sonnet 4.5, Agent SDK, and checkpoints
- 10:49Opus 4.5 and the role of sub-agents
- 12:05First long-running agent patterns
- 14:20Opus 4.6, Agent Teams, and server-side compaction
- 17:28State-of-the-art harness patterns
- 21:30Evaluating subjective output with rubrics
- 23:44Introducing the 'Planner' role
- 25:04The generator-evaluator contract
- 31:28Specificity in contracts and debugging traces
- 34:14Adjusting harnesses as models evolve
- 37:56How to build your own agent harness
- 39:01Key takeaways for long-running agents
- 40:05Q&A session
related talks
25mThe 3 Pillars of Autonomy
37mAI-Driven Developer Productivity at Scale
48mSupercharged Testing: AI-Powered Workflows with Playwright + MCP
21mDesigning in the age of the 10x engineer
55mKeynote: Exoskeletons, not Autopilots
15mDevelop at Idea Velocity
40mHow To Tame Your Agents?
45mAI agents in practice - beyond the prototype