Eradicate Flaky Tests

on YouTube ↗47m

" summary

Bold claim: eradicate flaky tests by three concrete moves: reduce UI tests, Specmatic-based runtime stubs for external dependencies, and Applitools Eyes visual AI instead of brittle functional assertions. Demo runs one automated UI flow that triggers 21 device/browser checks via the ultrafast grid, stubs dynamic user lists, edits a user and catches a missing button visually while functional locators still pass. Sample code and contract-driven stubbing show deterministic, scalable tests that cut flakiness and speed feedback.

AI-written summary · about

" chapters (21)

1. 0:00Intro
2. 0:33About Me
3. 4:11Challenges of E2E Test Automation
4. 6:33Why are Tests Flaky?
5. 8:46How to Reduce Flakiness in Tess
6. 10:06Product Quality
7. 12:59Test Scenario
8. 16:37Specmatic - Contract Driven Developmom
9. 17:41Use Specmatic as part of Test execrow
10. 22:01Bugs still Escape
11. 23:27Approach to Testing is incorred
12. 25:50Spot the Difference!
13. 27:39Pixel comparisons waste time with
14. 28:28Perfection Across All Screens, Browsers Viewports
15. 31:39Example Scenario using Specmatic
16. 38:22Make your Automation Intelligere
17. 38:52Reduce UI tests
18. 39:33Use Visual Assertions
19. 39:49Ultrafast Grid
20. 40:06Virtualize Dependencies at Runt
21. 40:33Resources

" related talks

• 44mTechniques to Eradicate Flaky TestsAnand Bagmar · Testing Talks Conference 2022 Melbourne
• 26mKeeping Your Tests LeanMeaghan Lewis · SeleniumConf Berlin 2017
• 51mRedefining test automationRichard Bradshaw · SeleniumConf London 2019
• 48mFlaky Tests 2.0Andrei Solntsev · DelEx Conf 2019
• 28mTest flakiness; methods for dealing with flaky testsJason Palmer · Assert(js) 2019
• 08mStop the Retries in Tests & Reruns of Failing TestAnand Bagmar · TestFlix 2020
• 51mUI tests are dead, long live UI tests!Aida Manna · Software Crafters Barcelona 2019
• 27mYour Tests Lack Vision: Adding Eyes to Your Automation FrameworkAngie Jones · JSConf Hawaiʻi 2020