Data Science Careers via A/B Testing
summary
Claims concrete changes: variance reduction techniques halve variance halves sample size and cut time to significance; pre-experiment covariates via Cupid and QAC using ML actually reduce noise in examples; design rules pick switchback for marketplaces and geo randomization for Jio DoDash Uber Lyft; operational checklist catches sample ratio mismatch, logging gaps, unit-of-analysis errors; a built cloud-native experimentation platform is described that makes tests reproducible, observable, cheap to run; portfolio advice names statmodel PMC and Qflow.
AI-written summary · about
chapters (12)
- 0:00Introduction and Session Overview
- 1:48Why AB Testing Matters for Your Career
- 3:12Experiment Design Patterns
- 5:19Advanced Experimentation Techniques
- 7:52Variance Reduction Techniques
- 11:41Common Pitfalls in Experimentation
- 12:42Accelerating Your Career with Experimentation
- 13:22Building a Cloud-Native Experimentation Platform
- 14:20Tips for Building an Experimentation Portfolio
- 15:02Interview Success Strategies
- 15:40Career Trajectory in Data Science Experimentation
- 16:52Conclusion and Action Plan
related talks
28mOptimizing your A/B testing roadmap
38mTesting, Testing: On Experimental Drift and Data Driven Product Design
25mMake a Difference with Simple A/B Testing
33mHow to A/B test with confidence
42mA/B testing with real-world data to evaluate ML model updates
43mLean QA: Automation, Monitoring, Experiments
42mTesting a Data Science Model
45mHow we killed 80% of features and increased outcomes of a/b testing by 100%