Re-thinking Regression Testing
summary
Claim: code coverage often reads ~85% while test redundancy sits near 25% and actual regression-detection falls to ~55%, so coverage is a weak proxy. Practical method: apply mutation testing — flip operators, inject first-order mutants, measure the mutation score as killed mutants over generated mutants. Use existing tools (heckle, mutant or custom engine), require passing tests, prefer low-order mutants but sample higher orders, run about 50 mutants for a reliable quick estimate, and fix tests where live mutants remain.
AI-written summary · about
chapters (11)
- 0:00Start
- 0:17Gold Sponsors
- 11:51competent programmer hypothesis
- 13:48coupling effect
- 14:47for regression analysis
- 16:51correlation code coverage
- 17:43correlation: likelihood of bugs
- 20:07correlation: mutation score
- 22:04passing tests
- 22:22mutants
- 23:50cycle
related talks
38mMutants to the rescue: How effective are your unit tests?
24mGain more confidence in your code with mutation testing
36mMutation Testing: Did my test break my code ? 🤔
41mMutation testing in practice
37mAre your tests good enough?
47mKill All Mutants! (Intro to Mutation Testing)
07mWho is Testing the Tests?
48mMutation Testing, or What Code Coverage Keeps Silent About