Generative AI Monitoring with PydanticAI and Logfire
summary
Demonstrates wiring PydanticAI agents to Logfire for end-to-end observability: agents call tools like get_current_date, HTTP calls via HTTPX are instrumented, and Logfire records spans showing the LLM requesting a tool, the function call, and the returned string. Structured outputs validate with Pydantic types and trigger a single retry on validation errors. Shows using MCP/Playwright for browser tools, running batch evolves (e.g., Nvidia case) and running local LLMs (Llama 3.2) with mixed success.
AI-written summary · about
chapters (9)
- 0:00Introduction to talk
- 1:27Creating a basic agent
- 6:04Monitoring with Logfire
- 8:32Enhancing tools with descriptions
- 10:19Structured outputs and validation
- 13:12Adding web search tools
- 17:49Model Context Protocol basics
- 20:21Testing agents with Evolves
- 23:40Q&A session
related talks
01h 04mBuilding AI Applications the Pydantic Way
24mDurable Agents: long running AI workflows in a flakey world
01h 21mAgent Optimization with Pydantic AI: GEPA, Evals, Feedback Loops
12mHuman seeded Evals
30mBuilding Reliable AI Agents: An Open Source Approach To Evaluation and Observability
48mHands-on guide to monitor your API-driven AI/LLM applications
38mEnhanced Log Analytics Using Generative AI
47mValidation and Observability of AI Agents