How to solve intermittent issues, aka Random Failures
summary
Explains three concrete levers: test design, logging, and tolerance. Recommends removing UI tests that duplicate lower‑level checks, implementing screenshots on failure (entire desktop) and timestamped logs to match app/CI errors, and using retries or suppression tags with expiry for known environmental flakiness. Gives failures that mattered: CI responses flagged tests at 2am, antivirus modal halts once a month, and a 30s “all green” build that merely rewrote results. Mentions using javascript latching or explicit waits for stale‑element/race conditions.
AI-written summary · about
chapters (5)
- 0:00Intro
- 3:42Intermittent Failures
- 5:34Why flaky tests happen
- 10:58Investigating flaky tests
- 14:53Tolerance
related talks
40mFinding Flakiness: The Real Reasons Your Tests Aren't Reliable
08mStop the Retries in Tests & Reruns of Failing Test
41mDeterministic Solutions to Intermittent Failures
36mFixing Flaky Tests Like a Detective
42mTesting on Thin Ice: Chipping Away at Test Unpredictability
39mStop the Retries in Tests & Reruns of Failing Tests
42mDon't Let Your Tests Flake Out
25mHow to ensure Testing Robustness in Continuous Testing Environments