Breaking Automation Rules for Better Efficiency and Coverage
summary
A team cuts full automation runtime from 47 hours to 2.5 hours and increases coverage by abandoning strict isolation: they replace repeated 10–25 minute environment resets (17 resets per run) with chained end‑to‑end scenarios, accept longer setup sequences, and assign a small dedicated automation team to own flaky results (previously a 20% flaky failure rate across three costly test rigs). The claim: breaking “always” rules and treating guidelines as heuristics yields far faster execution, easier debugging via richer logging, and broader, more realistic coverage.
AI-written summary · about
chapters (21)
- 0:00Introduction
- 1:07VDSL automation at Alcatel
- 3:14Better test automation principles
- 8:33Follow-up after principles
- 9:19Unexpected results from principles
- 13:59Three months later and the dramatic improvement
- 16:08Automation principles as heuristics
- 17:57Joel’s collocation principle
- 20:27Shift left vs end-to-end testing
- 22:59Regression test duration
- 25:07Making test failures clear
- 29:02Testing vs coding expertise
- 30:28Live automation connections
- 32:57Testing pyramid models
- 35:29Limiting verification duration
- 37:35Importance of 'it depends'
- 38:16Balancing automation and exploration
- 40:41Resetting environments vs client behavior
- 42:45Automation teams and ownership
- 43:49Testing frequency and pipeline
- 45:57Conclusion
related talks
36m5 Ways to Boost Automation Effectiveness
21mThe Essential Pillars of Automation Testing for Success
45mPlan Automation with 5% Rule: The Incremental Approach
51mRedefining test automation
14m3 Automation Testing Mistakes & How to Correct Them?
05m"Share of Experience : Test Automation"
01h 08mTest Automation != Less Testers || Faster Testing || More Time For ET
26mQuality not Quantity – Getting Value out of Test Automation