Effective AI Agents Need Data Flywheels, Not The Next Biggest LLM
summary
Claims a working system that uses Nemo microservices and Nvidia NIM to run a continuous data flywheel that curates feedback, retrains, evaluates and promotes models. A 70B router model hits 96% routing accuracy at ~26s first token while an out‑of‑the‑box 8B is 14% accuracy; with 685 curated examples fine‑tuned 8B and even 1B variants reach ≈96% or 94% respectively, yielding much lower latency and up to ~98% inference cost savings, so smaller models win once the flywheel is running.
AI-written summary · about
chapters (8)
- 0:00Introduction to AI agents
- 0:46Challenges in agent scaling
- 2:36Data flywheels for agents
- 3:46NeMo microservices
- 5:27Data flywheel architecture
- 6:41Case study: NVInfo agent
- 9:27Case study results
- 14:31Data flywheel framework
related talks
46mArchitecting Efficient Agents with Semantic Routing and Mixture-of-Tools
14mReliable Agentic AI, the Open Way
20mFireworks AI: Go from Hype to High - Value AI
28mNavigating LLM deployment: Tips, tricks, and techniques
20mArchitecting AI: Efficient Models, Everywhere
33mBuild Better AI with First-Party Data and Fine-Tuning
36mLLMs and AI agents demystified
36mProduction-Ready AI Agents: From LLMs to Small Language Models