How I deleted 95% of my agent skills and got better results
summary
A harness named Case is rebuilt on Pi with a TypeScript state machine enforcing gates and evidence collection; tests are verified by SHA-256 hashing of actual output to stop Claude faking tests. Generating 10,000 lines of doc‑derived skills bloats evals to 68 minutes and drops accuracy, so pruning and rewriting yields 553 lines of gotchas and cuts eval time to 6 minutes, effectively deleted 95% of the noise. Enforce with code and measure with evals.
AI-written summary · about
chapters (13)
- 0:00Introduction
- 1:22The challenge of context switching with agents
- 2:33Introducing Case: A harness for agentic workflows
- 3:33Rebuilding with a TypeScript state machine
- 4:45The critical importance of evidence-based verification
- 5:59Applying agentic principles to the WorkOS CLI
- 7:44Lessons in documentation: Generating skills from docs
- 8:52Why more data (10,000 lines) led to worse performance
- 9:36The impact of using evals to measure accuracy
- 10:40Key takeaway: Enforce with code, not just prompts
- 12:41Treating failures as bugs in the harness system
- 14:39Advice for building agentic-ready products
- 16:01Final summary: Replacing trust with evidence
related talks
29mStop Prompting, Start Composing: Agent Skills in Practice
02h 04mShip Real Agents: Hands-On Evals for Agentic Applications
27mAgentic Coding with Discipline & Skill: Practical Techniques and Tips
32mMy toxic relationship with AI - Abeba NGWE
07h 47mAI Agents Conference 2026 - Day 5
21mBuilding Great Agent Skills: The Missing Manual
01h 00m10 tips to level up your ai-assisted coding
40mHow To Tame Your Agents?