How to build a continuous evaluation pipeline for multi-agent systems
summary
Shows a runnable codelab that builds an automated regression pipeline for distributed agents using Gemini Enterprise Agent Platform, Cloud Run and the Gemini evaluation SDK, with tests run locally or against shadow deployments tagged by commit. Implements code-based and model-based graders, measures trajectory precision and tool‑use quality, runs multiple LLM judgments per case to reduce noise, and demonstrates a course‑builder multi‑agent that passes the lab’s hallucination and response‑match checks.
AI-written summary · about
chapters (11)
- 0:00Introduction
- 2:55The problem with vibe checks
- 7:10Discovery vs. defense
- 13:40Evaluation metrics
- 21:31Graders and judging
- 29:08Agent platform tools
- 34:20Implementing tests
- 38:46Executing evaluations
- 48:03Rubrics and analysis
- 57:35CI/CD and shadow deployment
- 1:07:55Q&A and conclusion
related talks
30mLive Demo – Develop and Deploy a Trustworthy Multi-Agent System
47mBuild Trustworthy AI Agents powered by Evals
45mCreate and deploy your first agent with Google Cloud Gemini Enterprise
47mHow to Build Enterprise-Grade AI Agents Using Robust Evaluation
49mLet's build an AI agent
24mScalable GenAI in Production
30mBuilding Reliable AI Agents: An Open Source Approach To Evaluation and Observability
02h 04mShip Real Agents: Hands-On Evals for Agentic Applications