The 100-Tool Agent Is a Trap
summary
Fat-agent designs load hundreds of tool schemas into every prompt and show concrete failure modes: accuracy falls from ~78% at 10 tools to ~13–13.6% at ~741–1041 tools, time-to-first-token exceeds 5s around 500 tools, and prompts can balloon to 127,000 tokens of tool text. A Semantic Tool Router embeds tool descriptions in a vector DB (Chroma/Pinecone/Quadrant), retrieves top K and injects only those schemas, yielding up to a 99% reduction in tool-context tokens and stable >83% accuracy with K=5 default.
AI-written summary · about
chapters (19)
- 0:00Introduction
- 1:48Speaker background
- 2:53Fat agent explained
- 4:19Accuracy collapse with many tools
- 5:45Latency and cost of fat agents
- 7:29Fat agent vs semantic router
- 9:14When to use semantic routing
- 11:01How semantic routing works
- 12:44Just-in-time context injection
- 13:50Benchmarking results
- 16:00Semantic routing pattern
- 17:46Code walkthrough
- 19:11Semantic routing examples
- 20:37Production checklist
- 22:25Existing projects
- 24:32Trade-offs of semantic routing
- 25:58Key takeaways
- 27:22References and resources for further learning
- 28:05Closing
related talks
01h 01mAI Agents In-Depth – Function Calling, MCP and Tool Use Under the Hood
45mAI agents in practice - beyond the prototype
46mArchitecting Efficient Agents with Semantic Routing and Mixture-of-Tools
19mYour Biggest User Isn't Human: Designing DevTools for AI Agents
23mBuilding Agent Interfaces: Lessons from Chrome DevTools (MCP) for Agents
21mHow agent o11y differs from traditional o11y
39mFrom Demo to Production: Why Agentic AI Systems Fail - and How to Fix Them
25mReduce AI Agents Costs with Semantic tool Selection