Make your tests fail
summary
Uses the Carrotsearch Randomized Testing framework that Apache Lucene and Elasticsearch adopt, generating inputs from reproducible seeds and varying JVM versions and configs. It claims concrete wins: uncovering thread leaks, JVM bugs and even index corruption that lead to data loss, and catching locale and environment edge cases. Adopting it initially makes CI flaky as many hidden bugs surface, requires weeks of fixes and cultural changes around quick triage, but yields broader coverage and fewer surprises in production. Carrotsearch Randomized Testing
AI-written summary · about
chapters (5)
- 0:00Start
- 6:05Test Timeouts
- 14:01What Impact Does that Have on Development Culture
- 19:18Teach Developers How To Write Tests That Are Easy To Reproduce
- 27:08Build a Randomized Test
related talks
45mRandomized testing: Gotta Catch ‘Em All
14mChance Makes Good Tests
42mTesting on Thin Ice: Chipping Away at Test Unpredictability
40mCatching More Bugs with Fewer False Alarms
55mThe art of living with broken things
49mInside the Elastic: Testing a Well Known Open Source Stack
30mApplying AI to testing
47mTesting on Thin Ice: Chipping Away at Test Unpredictability