The Build That Cried Broken: Building Trust in Your Continuous Integration Tests
summary
Team fixes a flaky CI by triaging 4,000 tests into a trustworthy and an investigational build, logging each test result to a small DB to surface flaky and newly failing cases, and enforcing weekly rotation to monitor tests. They remove hard sleeps, prefer stable element identifiers, use API “shortcuts” instead of full UI flows, delete-and-rewrite hopeless tests, and gate new tests in a private build until ~10 green runs so the reliable build stays <1.5% unstable even with 4,000 tests.
AI-written summary · about
chapters (16)
- 0:00Intro
- 0:16What is Continuous Integration
- 1:40The shepherd boy who cried wolf
- 9:40How to avoid flaky tests
- 15:49Story time
- 17:12Test monitoring
- 18:31Test database
- 19:32Farmer and the Stork
- 20:42One bad apple spoils the whole bunch
- 21:23Separate the tests
- 22:55Test annotations
- 23:31Test backlog
- 24:09Delete and start over
- 25:45The dog in the hair
- 27:38Private Bills
- 29:08Conclusion
related talks
40mThe Build That Cried Broken: Building Trust in Your Continuous Integration Tests
32mBuilding Confidence Together: Continuous Integration & Testing
38mScaling Automated Testing & Architecture from Startup to Enterprise
46mFlaky, Fragile, and Forgotten: The Fall of a Test Automation Project
45mRunning Tests in Continuous Integration
39mBuilding The Blocks of Trust In Automation
01h 15mTaking Control Of Your Test Environment
24mHow to solve intermittent issues, aka Random Failures