Testing and Assuring AI-Based Systems
summary
The talk gives concrete testing steps and an assurance framework rather than hype: a 2i-built Assure AI checklist that scores systems by accuracy robustness explainability performance, concrete test levels like model testing and data testing plus component/integration/end-to-end/UAT and live monitoring, and specific techniques — statistical comparison of synthetic vs real data, K‑fold cross‑validation, AB/parallel/model‑back testing, metamorphic tests, adversarial/poisoning checks and explainability tools — with metrics to decide production readiness.
AI-written summary · about
chapters (17)
- 0:00Testing AI systems
- 1:07AI testing challenges
- 3:39AI-based system problems
- 5:26Test levels for AI
- 6:10Component testing data
- 7:37Data testing and bias
- 11:14Integration with probabilistic systems
- 12:24Model testing and stats
- 14:12End-to-end testing
- 14:54User acceptance testing
- 15:35Monitoring AI drift
- 17:00AI security testing
- 18:05Explainability testing
- 19:09Advanced AI testing
- 22:16AI quality frameworks
- 25:53Resources and closing
- 26:35Q&A
related talks
32mQuality engineering in the AI age: From testing AI to leveraging GenAI
43mAI for Testing – Panel Discussion
01h 05mPanel Discussion: Building Confidence in AI-Driven Testing Decisions
20mKeynote: Testing In The Era of AI - From QA to Trust Engineering
37mAI-Assisted Unit Test Development
30mAI in QA. Rise of the Machines?
01h 05mPanel Discussion - The Future of Testing: Impact of AI in Quality Assurance and Beyond
31mAgentic AI Changed What We Test, And How We Test It