Computer Vision in UI testing
summary
Explains how Computer Vision can be used in test automation to detect and interact with UI elements that are not represented in HTML/XML—such as graphs, charts, maps and schematic visuals—and demonstrates applying CV techniques to locate and verify those visual elements in UI testing.
AI-written summary · about
chapters (22)
- 0:00Start
- 1:27How do we test applications today?
- 4:11What about the screenshot comparison?
- 4:50Car position. Hamming distance
- 5:18Zoom out. Hamming distance
- 6:09Samples. Hamming distance
- 7:44Template matching. Solution for 90% of cases
- 9:18Template matching methods
- 10:17Multiple objects matching
- 12:14Understanding features
- 15:44Descriptors
- 16:43Feature matching
- 16:53Cross check & Ratio test
- 20:24Brute Force matching
- 22:35CV based test framework
- 24:09Core scripts. Find templates on image. Example
- 24:54Core scripts. Find templates. Sparsing points
- 26:11Core scripts. Find objects on image. Example
- 27:38Core scripts. Find objects. Negative
- 28:15Let's run the test!
- 29:35Google Fit tests. Steps
- 32:00Google Fit tests. Cardio + Activity
related talks
41mComputer Vision as a technology to test Navigation UI in cars
27mVisual UI Testing – Testing the Last Mile
30mHow AI is Transforming Test Automation
48mChallenges in UI Automation and How to Solve Them
38mYour Tests Lack Vision: Adding Eyes to Your Automation Framework
36mVisual Validation - The Missing Tip of the Automation Pyramid
36mVisual Validation - The Missing Tip of the Automation Pyramid
43mYour Tests Lack Vision: Adding Eyes to Your Automation Framework