Architecting Efficient Agents with Semantic Routing and Mixture-of-Tools
summary
Claims a semantic router (VLM envoy proxy + small classifier + keyword/embedding signals) routes queries to the right model and cuts cost and time—benchmarks show 47% latency reduction, 48% token reduction and ~10% accuracy uplift. Uses tool‑RAG (Chroma DB vector retrieval of MCP tool definitions) to avoid the “junk drawer” (26 MCP tools can consume ~4,600 tokens). Demo runs a Llama.cpp model on DGX with Gemini 2.5 Pro as critic, a Yahoo Finance MCP, MLflow tracing, thousands of synthetic scenarios and a 35GB runtime; the architecture and reflexive metacognition loops succeed in reducing hallucination and improving auditable decisions.
AI-written summary · about
chapters (10)
- 0:00Efficient agentic systems intro
- 1:47Agent design challenges
- 4:14Helix architecture layers
- 6:01Semantic routing for reasoning
- 11:13Tool selection to avoid failure
- 15:09Metacognition for governance
- 20:45Live financial agent demo
- 32:17Agent tracing with MLflow
- 35:51Agent design checklist
- 39:00Resources and closing
related talks
36mProduction-Ready AI Agents: From LLMs to Small Language Models
53mFrom Mixture of Experts to Mixture of Agents with Super Fast Inference
40mSurviving the Agentic Hype with Small Language Models
37mRAG, Tooling, MCP, Agents ... and What is Missing?
25mReduce AI Agents Costs with Semantic tool Selection
29mBuilding Modular AI Agents with LangGraph, MCP, and
20mEnhancing RAG with Multi-Agent Integration
01h 03mAgents Everywhere