Confidence - Correctness: The Agentic Validation Loop
summary
Claim: agentic pipelines grade their own work so confidence often equals false certainty. The talk demos a closed-loop Agentic Validation Loop and an external validation tool Keen CLI that emits Playwright, runs simulators, auto-heals locators and produces shareable evidence. Demo numbers: Keen ingests a PRD, generates 24 acceptance criteria, 11 scenarios and 5 gaps, runs tests against 99acres, records a 403 access denied and reports 13% acceptance criteria passed. Architecture enforces five validation stages, evidence as first-class, and separates generator from grader.
AI-written summary · about
chapters (10)
- 0:00Why Can AI Agents Be Confident and Still Be Wrong?
- 4:18How Has Agentic AI Changed Software Testing?
- 6:51How Can AI-Generated Changes Break Existing User Flows?
- 8:22What Does a Reliable AI Agent Validation Framework Need?
- 12:53How Do You Prove What an AI Agent Actually Tested?
- 14:58How Kane CLI Validates AI-Generated Code
- 21:09From PRD to Test Cases: Agentic Validation in Action
- 24:44How to Measure Acceptance Criteria Coverage
- 27:48Why Evidence Matters More Than a Green CI Check
- 29:51How to Handle Hallucinating Agents, Context, and Messy PRDs
related talks
23mBeyond Tests What to Verify in AI Generated Code
02h 04mShip Real Agents: Hands-On Evals for Agentic Applications
35mThe Agentic AI Engineer
51mMake Quality Executable: Verification Gates for Agentic Engineering at Scale- 02h 04mShip Real Agents: Hands-On Evals for Agentic Applications
20mDark Factories: Verification-First Agentic Engineering
46mEnhancing Agent Reliability Through Evaluation-Driven Development
33mEnsuring quality in an AI-driven world