SW Testing: Can ML save us?
summary
Argues machine learning can reduce GUI test maintenance by replacing brittle pixel diffs with Applitools screenshot comparison that ignores minor changes and matches text and layout, but needs hundreds to thousands of labeled images often gathered via AWS Mechanical Turk or transfer learning (Test.ai approach). Shows unsupervised bag-of-words clustering of thousands of logs to group failures and surface anomalies, and uses reinforcement learning like King’s Candy Crush bot to drive exploratory tests and catch crashes when the system is fast and observable.
AI-written summary · about
chapters (17)
- 0:00Intro
- 1:49What is software testing?
- 2:12The problems with testing today
- 3:53The promise of Machine Learning for testing
- 8:40Problem: GUI testing is a mess
- 9:10GUI Regression testing
- 13:12Test abstractions in GUI-testing
- 15:07Example: abstract GUI testing
- 16:11GUI-abstraction with Supervised Learning
- 24:32The ingredients of a clustering solution
- 25:24The bag-of-words model
- 27:44Characteristics of Unsupervised Learning
- 28:31Unsupervised Learning: Criteria for success
- 30:40Testing Candy Crush with Reinforcement Learning
- 34:19How to succeed with Reinforcement Leaming
- 35:40Related: Fuzzing
- 38:18Testing machine learning systems
related talks
16mTransforming Test Automation with AI and Machine Learning
28mML-Driven Test Intelligence at Scale - What Works What Fails and Why It Matters
31mTesting Machine Learning Models
01h 04mWhat Testers Need from Machine Learning and Artificial Intelligence
50mPractical Applications of ML in Software Testing
53mUnleashing the Black Box: Testing Machine Learning Systems
40mMachine Learning helps to work with your test automation
38mHow to Test a learning system