Visual regression testing: your strategy's blind spot

on YouTube ↗40m

" summary

All green tests still miss pixel bugs so screenshot testing catches visual regressions that functional tests never see, demonstrated by a translator app where a color change from red to blue breaks UI expectations but not logic. The demo uses WireMock+Maestro+ImageMagick to fake servers, drive the UI, crop system bars and produce baseline, actual and diff directories, stores baselines in a Cloudflare/S3 bucket, and flags diffs in CI; it works but requires nontrivial setup and flakiness handling.

AI-written summary · about

" chapters (12)

1. 0:00Testing blind spot
2. 5:03Testing pyramid limits
3. 6:48Visual regression testing
4. 8:14Benefits of visual testing
5. 10:23Implementing visual tests
6. 13:12Baseline storage pitfalls
7. 17:29Emulators vs device farms
8. 20:17Deterministic screenshots
9. 22:44Demo setup with WireMock
10. 28:06Running the demo
11. 32:40Wrap-up advice
12. 35:52Q&A

" related talks

• 37mThe Colorful World of Visual Test AutomationOnur Baskirt · Selenium Camp 2017
• 27mVisual UI Testing – Testing the Last MileCraig Cavalier · React Boston 2019
• 53mPainless visual testingGojko Adzic · NDC London 2018
• 20mVisual testing with Visual Regression TrackerPavel Strunkin · Appium Conference 2021
• 12mAutomation of GUI testing using visual regression testingAudun Wigum Arbo · NDC Oslo 2023
• 39mSpot the difference: automating visual regression testingViv Richards · SeleniumConf Chicago 2018
• 43mYour Tests Lack Vision: Adding Eyes to Your Automation FrameworkAngie Jones · Full Stack Fest 2019
• 21mVisual Regression Testing: From a Tool to a ProcessNikhil Verma · jsDay 2017