Beyond Tests What to Verify in AI Generated Code
summary
Claims an invariant-driven workflow that catches bugs testing misses and demonstrates it: an agentic demo uses Claude plus an internal MCP transaction analyzer to find a race in reserve_items that would allow double reservations, detects a lean() omission in Mongoose queries that can blow memory under load, and exposes a combinatorial validation space of 2500 permutations. The invariant taxonomy (scope, type, remediation) maps to runtime Specification Registry checks, pre-save hooks or CI gates and the agent auto-fixes or blocks merges in the demo.
AI-written summary · about
chapters (13)
- 0:00Introduction by speakers
- 1:48Why tests miss critical bugs
- 3:37Three untestable bug categories
- 6:26Demo of race condition issue
- 7:49Demo of performance flaw
- 8:53Demo of state validation complexity
- 10:18Invariance verification explained
- 12:04Invariance in AI-generated code
- 15:53Fixing performance anti-pattern
- 17:37State consistency invariants
- 19:42Action plan for developers
- 21:06Roadmap for implementation
- 22:11Closing and Credential.ai
related talks
01h 07mHappiness U-Curve: Navigating the AI Validation Bottleneck with Conformance Testing and(…)
32mEscape the Try-Again Loop: Verifying AI-Generated Code
36mConfidence - Correctness: The Agentic Validation Loop
32mQuality engineering in the AI age: From testing AI to leveraging GenAI
54mYou Can't assertEquals an Agent: A Tester's Guide to Agentic Quality
32mCan We Trust AI-Generated Code?
50mPanel Discussion: What Can Go Wrong With AI in Testing?
47mTDD & Generative AI: A Perfect Pairing