Testing in Production
summary
Claim: test in production by default, not staging; staging often costs over $100k/year and still diverges from prod. Uses feature flags via Split.io to target internal users, automation tests run by CI jobs, alerting integrated with job scheduler, back‑end booleans like is_test_user to exclude test traffic, and production canaries to roll out small percentages. Built test robots and targeted test user sets, deploys behind flags, and measures: higher confidence, faster velocity, fewer user-facing bugs.
AI-written summary · about
chapters (31)
- 0:00Intro
- 1:00What is testing in production
- 2:00What is important to remember
- 3:00Staging environments are expensive
- 4:00Dont always match production test results
- 5:00Staging is broken
- 6:00Testing in production
- 7:00Trust
- 8:00Risks
- 11:00Feature Flags
- 12:00Split UI
- 13:00Targeting Rules
- 14:00Automation Framework
- 16:00Naming Convention
- 17:00Test Guidelines
- 18:00Alerts and Deployment
- 19:00Risk Mitigation
- 20:00Summary
- 21:00Questions
- 22:00Staging Environment
- 23:00Checkout Flow
- 24:00ThirdParty Flow
- 25:00How to Get Started
- 26:00How Much Do You Use Database Routing
- 27:00Working with a Third Party
- 29:00Load Tests
- 30:00Use Canary Releases
- 31:00Feature Flags and Complexity
- 33:00Feature Flogging
- 34:00Number of Parallel Feature Flags
- 35:00Thank you
