Test State Pattern – For a Sane & Stable Automation
summary
Claims a concrete reduction in flakiness at WixEngineering by running tests through a server-driven lifecycle: Test State Pattern with states like beta, stable, flaky, failed beta and optional super stable. Implements a Java client and REST API to report thousands of run results, an algorithm that promotes a test after X consecutive passes (demo uses 35 runs), and side-runs for flaky/failed tests. Live demo shows the system and UI working; source for the runner is promised and integration with other frameworks is supported.
AI-written summary · about
chapters (9)
- 0:00Introduction
- 1:45Endtoend tests
- 4:30Beta test
- 15:39Superstable state
- 17:45Configuration
- 20:14Model
- 21:57Live Demo
- 31:37Questions
- 39:27Automation Manager
related talks
45mTesting the untestable PART II – patterns, use cases and practices
28mTest flakiness; methods for dealing with flaky tests
27mWrite Fewer Tests! From Automation to Autogeneration
47mThe Valley of Success
46mThe state of the Art in Tackling Flaky Tests
44mTesting the untestable - patterns and use cases analysis
47mEradicate Flaky Tests
25mTest Craftsmanship: Clean code practices for Test Automation