Testing in prod (without pain and humiliation)
summary
Describes a production testing approach where real users implicitly help validate and tune algorithms without being aware. Covers practices for running tests in production—how user interactions serve as test signals and how the system uses that data to customize and improve algorithms. Discusses the workflow and mechanisms used to implement and manage such in-production testing.
AI-written summary · about
chapters (19)
- 0:00Testing in prod intro
- 3:08Why test in production
- 10:17Testing-in-prod mistakes
- 11:02Feature toggles for safety
- 13:31Canary releases
- 15:01Traffic replication
- 18:59Parallel code execution
- 21:49Tools for quality feedback
- 23:57System monitoring metrics
- 27:08Case study: canary releases
- 31:00Algorithm quality metrics
- 34:12Safe rollbacks
- 36:22Cleanup after testing
- 37:49Q&A: business metrics
- 41:53Q&A: DB migration safety
- 45:06Q&A: prod data testing
- 46:55Q&A: risks and experiences
- 48:22Q&A: multi-feature management
- 51:07Summary
