Human seeded Evals
summary
Introduces using Pydantic models as executable contracts so the LLM returns structured output, then feeds validation-as-feedback back into the model to correct errors; demo uses Pydantic AI with Gemini and an observability backend Logfire, showing a simple DOB validator that causes two calls to the model and then a successful retry. Also demos type-safe agent tools for record/retrieve memory, shows a failed run then a successful run with trace timing and cost.
AI-written summary · about
chapters (10)
- 0:00Introduction
- 0:44Building reliable AI apps
- 1:05Type safety in AI applications
- 2:10What is an AI agent?
- 3:34Minimal agent example
- 5:59Validation error handling
- 7:03Type safety benefits
- 8:09Agents with typed tools
- 9:53Memory tool demo
- 11:42Observability and pricing
related talks
01h 04mBuilding AI Applications the Pydantic Way
01h 21mAgent Optimization with Pydantic AI: GEPA, Evals, Feedback Loops
47mBuild Trustworthy AI Agents powered by Evals
37mDr. Egor Kraev | AI Track | Compass AI & Data Summit 2025
30mGenerative AI Monitoring with PydanticAI and Logfire
57mEvaluating AI Agents in production with Python
30mBuilding Reliable AI Agents: An Open Source Approach To Evaluation and Observability
25mEngineering Better Evals: Scalable LLM Evaluation Pipelines That Work