Yellow is the new Green!
summary
Claims that pass/fail binary regression reporting drives unsafe behaviour and proposes yellow metric as the real release target: tests report step/assert-level results, no inconclusive results are allowed, and every failing assertion must link to an open defect triaged by the product owner with none prioritized for the imminent release. Demonstrates a concrete implementation (SoftASearch/soft asserts in MAX) that marks assertions as failed but continues execution so reports reflect multiple failures without hiding known bugs.
AI-written summary · about
chapters (6)
- 0:00Introduction
- 1:08Testing green expectations
- 6:27Is green realistic?
- 13:04Regression testing pitfalls
- 23:55Yellow as new green
- 28:53Conclusion
related talks
31mMeasure it or it didnt happen
21mHow to Make Releases Safer in Baidu
29mCode-Yellow: Helping Operations Top-Heavy Teams the Smart Way
40mVisual regression testing: your strategy's blind spot
01h 15mTaking Control Of Your Test Environment
26m"Tests as Classifiers"
22mFail Fast, Fail Often - Make Safety a Prerequisite for your Team
55mThe art of living with broken things