Eliminating Inconsistent Test Failures
summary
Session lists concrete fixes: diagnose Capybara and Cucumber flakiness by reproducing CI state, capture DB with fixture_builder, stub third‑party calls with VCR/webmock, and run a tiered build that falls back to sequential tests; parallel_test plus cucumber‑rerun increases failures. A Mutex for request handling and a tiny prototype gem rspec_manual_order are built to binary‑search test pollution. A pair spends three months; random failures drop to under five percent and the tiered approach works.
AI-written summary · about
chapters (27)
- 0:00Start
- 3:25Continuous Integration
- 3:43The usual suspects
- 4:53Speed daemons
- 5:52Mysterious benefactor
- 7:22Reduced trust
- 7:38Run it again
- 10:01What Goes Wrong?
- 10:39Cucumber Capybara
- 11:03Browser
- 13:07Race conditions
- 15:07Threads & Database connections
- 15:57DB-backed object solutions
- 16:09Immutable database
- 20:38Policy failure
- 22:10The Data
- 22:42The Network
- 23:22The "external" build
- 24:13Test Pollution
- 24:29And the problem is...?
- 26:21Re-use your data
- 27:45Tighten feedback loops
- 28:25Binary Search
- 29:11Step
- 31:43Eliminating Random Test Failures
- 32:14F*cking Snowflakes
- 32:30Don't Give Up
related talks
29mCommon Mistakes that make test cases flaky and how to avoid them
36mFixing Flaky Tests Like a Detective
41mDeterministic Solutions to Intermittent Failures
36mIt's not your test framework, it's you
31mGoing on a Testing Anti-Pattern Safari
21mTaming Flaky Specs: Diagnosing Test Failures in Ruby
36mIt's not your test framework, it's you
16mBlack Box Testing with RSpec and Capybara