Don't Let Your Tests Flake Out
summary
Explains two root causes of flakiness: leaked shared state (globals, constants, class vars, callbacks, even 50 lingering event subscriptions) and brittle comparisons (floating point and time round‑trips, timezones, daylight savings). Recommends concrete fixes: isolate objects or reset state centrally, freeze/advance time with Timecop/time travel, use anonymous classes or appraisals for metaprogramming, expose and join threads, add verbose/seeded runs and logging. Ships three tools that work in practice: test‑bisect to find the minimal failing sequence, Hometown to trace object instantiation, and minitest‑stat to detect thread leaks.
AI-written summary · about
chapters (13)
- 0:00Start
- 3:09Globals
- 5:01'Constants
- 5:31Class Variables
- 6:19Events and Callbacks
- 11:10Floats
- 11:43Time Roundtripping
- 13:42SAVING DAYLIGHT
- 16:19Reset Consistently
- 19:47Metaprogramming
- 23:45Always Be Logging
- 25:56Method Order
- 28:46Threading
related talks
15mHow to (not) write a flaky test
29mCommon Mistakes that make test cases flaky and how to avoid them
42mTesting on Thin Ice: Chipping Away at Test Unpredictability
42mFlaky tests are a pain
46mThe state of the Art in Tackling Flaky Tests
21mTaming Flaky Specs: Diagnosing Test Failures in Ruby
31mDon’t Blink! How We Got Rid of Flaky Tests
41m0Flake - Reaching reliable non-flaky tests