Quality engineering in the AI age: From testing AI to leveraging GenAI
summary
Testing AI systems adds two new levels: input-data testing and explicit ML-model testing, alongside traditional UI/API/database checks; techniques shown include back‑to‑back, metamorphic and A/B testing. An experiment using "close copilot" and "jactivity" finds AI-generated test cases cut creation time and improve structural consistency but produce moderate ambiguity and need human oversight for complex scenarios; scoring uses three reviewers across multiple user stories. A live demo with play MCP opens a browser, handles a 409 conflict by retrying and succeeds for short flows but struggles with visual misreads, long multi‑state workflows, hallucinations, reasoning errors and data‑privacy risks.
AI-written summary · about
chapters (9)
- 0:00Introduction to AI in QA
- 1:07Testing AI-based systems
- 8:13New test levels for AI systems
- 13:39Key takeaways from testing AI
- 14:21Using GenAI to test software
- 15:50Experiment on AI-generated tests
- 20:13GenAI for exploratory testing
- 28:10Risks of using GenAI in testing
- 30:42Conclusion and final thoughts
related talks
55mAI-First Testing
41mSurviving and Thriving with AI in QA
44mHow AI is Exposing the Existential Crisis of Software Testing
55mSurviving the AI Testing Apocalypse
41mAI-Augmented Testing: How Generative AI, Prompt Eng Turn Testers Into Superheroes
29mQA vs AI: Friends or Foes?
43mAI for Testing – Panel Discussion
51mHow to effectively test your AI applications?