Solving Context Bloat: Semantic Tool Routing in Multi-Server MCP Environments
summary
Concrete setup: a TypeScript Volcano agent behind a Kong AI gateway calls OpenAI (GPT‑4) and registers MCP servers via a catalog. Adding one MCP server jumps a simple 20‑token prompt to 100 tokens; two more servers blow it to ~3,500 tokens. A registry plus an MCP gateway applies identity‑aware policies (Alice vs Bob reduces tokens from ~3,000 to ~1,300), and the gateway performs semantic routing and just‑in‑time tools injection to cut cost and hallucinations.
AI-written summary · about
chapters (13)
- 0:00Context load in MCP
- 3:34MCP promise vs reality
- 10:11Demonstrating context load
- 15:44Consequences of context bloat
- 18:53Just-in-time tool provisioning
- 21:25MCP gateway pattern
- 24:33Registry for MCP discovery
- 28:49LLM gateway guardrails
- 31:20Architecture overview
- 35:29Demo registry routing
- 41:05Token usage scenarios
- 44:55Dynamic tool updates
- 45:39Recap and future
related talks
23mMCP = Mega Context Problem
28mSmarter MCP Servers: Using a Graph to Solve the Context Window Problem
30mComposable AI: Building Next-Gen AI Agents with MCP
33mThe Model Context Protocol: Connecting AI to Everything
50mBeyond local tools: Dive into the Model Context Protocol (MCP)
17mBuilding Production-Ready MCP Servers: From AI Agents to Secure Enterprise APIs.
45mAgents, Tools, and MCP, oh my! Next-level AI concepts for developers
44mBeyond the Prompt: How MCP Connects Language Models to Real Tools