Your Tests Aren’t Flaky, You Are!
summary
Argues that flaky checks come from design and people choices not Selenium or the internet, cites Google’s analysis (bigger checks show more failure but size is symptom), and gives concrete failures like 23 tests passing while the site is down to prove non‑determinism. Recommends separating state setup from the algorithm, writing and revisiting codified oracles, keeping reporting out of the build, making tests fail locally, and avoiding quarantine layers that hide problems.
AI-written summary · about
chapters (16)
- 0:00Intro
- 2:22Why we automate
- 4:07Signs of flakiness
- 8:24Its gonna get worse
- 12:54The flakiness problem
- 14:40Use it as a trigger
- 15:31State
- 18:02Algorithm
- 20:31codified Oracles
- 23:11Reporting
- 24:47Execution
- 26:15deterministic
- 28:06decision making
- 29:23other options
- 29:34ministry of testing
- 30:03funny calendar
related talks
40mFinding Flakiness: The Real Reasons Your Tests Aren't Reliable
01h 09mFlaky Tests: The Method
46mThe state of the Art in Tackling Flaky Tests
42mDon't Let Your Tests Flake Out
15mHow to (not) write a flaky test
42mFlaky tests are a pain
39mTen things you heard about testing that might be wrong
42mTesting on Thin Ice: Chipping Away at Test Unpredictability