Test Management Ninja
summary
A small internal tool called Test Management Ninja automates classification and reliability for about 150 tests (snapshot over 14 days), running each new test 40 runs and promoting it if pass rate exceeds a 90% threshold. Test results flow into an open source datastore called out where the tool excludes environmental and infra exceptions, computes reliability, and moves tests between buckets like smoke, regression and “tests under test,” creating tickets when flakiness requires human fixes.
AI-written summary · about
chapters (6)
related talks
17mContinuous Testing at Scale
21mScaling Test Automation: Building Reusability for 10x Speed and Efficiency
41mThe Tester’s Guide to Quality
26mKeeping Your Tests Lean
34m1.5 Billion Tests Per Day
56mTestOps at scale
39mTrust Issues With UI Automation
49mHow to Optimize Automated Testing with Everyone's Favorite Butler
