The Art & Joy of Testing in Production
summary
Argues staging is unreliable and expensive—staging can cost up to 30% of project cost—and shows concrete alternatives: synthetic production checks that create test users in production to measure end-to-end sync latency and SLOs, a bot that buys then cancels orders, and traffic mirroring (he calls it tab compare/diffy) to run integration, performance and config tests against a deployed version. Adds controlled release practices—canaries, focused rings, feature flags—plus observability (logs, metrics, traces) and chaos experiments to validate resiliency and lower MTTR.
AI-written summary · about
chapters (24)
- 0:00Intro
- 2:51IS YOUR PRODUCTION ENVIRONMENT WORKING RIGHT NOW? DO YOU KNOW?
- 5:22WHEN WAS THE LAST TIME YOU RELEASED TO PRODUCTION? HOW DID YOU CHECK IT WAS SAFE?
- 6:35STAGING ENVIRONMENTS FALSE SENSE OF TRUST
- 11:25STAGING ENVIRONMENTS - FALSE SENSE OF TRUST FOCUS ON TRUSTING PRODUCTION
- 12:21STAGING ENVIRONMENTS EXPENSIVE
- 15:01STAGING ENVIRONMENTS NATURAL HANDOVER POINT
- 16:26STAGING ENVIRONMENTS BOTTLENECK
- 18:08VERIFY IN PRODUCTION
- 19:17USER SYNC SERVICE
- 22:39E-COMMERCE
- 27:40NEW VIEW ON TESTING
- 28:46TESTING IN PRODUCTION SPLIT DEPLOY & RELEASE
- 30:04INTEGRATION TESTING
- 31:24TAP COMPARE
- 32:19LOAD TESTING
- 33:55CONFIG TESTING
- 36:01CANARY RELEASES
- 37:13FOCUSSED CANARIES
- 38:01RING BASED RELEASES
- 41:48FEATURE FLAGS
- 44:11IMPACTED USERS?
- 46:52FOCUS ON RESILIENCY
- 49:05RECAP
