Did Your Rollout Actually Work?
summary
Demonstrates that naive pre/post and standard two‑way fixed‑effects give biased answers for staggered rollouts and that an imputation / staggered DiD approach recovers cohort‑level causal trajectories. Uses Python tooling—CausalPy and Bambi/PyMC—to produce event‑study plots with uncertainty and a reproducible notebook. Finds per‑cohort heterogeneity: urban and suburban waves show clear positive ROI while the third wave loses money, yielding a net lift around £2M/year; provides code to reproduce those numbers.
AI-written summary · about
chapters (12)
- 0:00Introduction
- 3:12Randomized trials rarity
- 7:12Store UK loyalty case
- 12:10Revenue analysis limits
- 17:08Bayesian two-way fixed effects
- 19:19Model assumptions flaws
- 21:28Event-time and trajectory effects
- 24:21Bias in difference-in-differences
- 27:56Staggered DiD estimator
- 30:50Cohort ROI and impact
- 32:34Avoiding this approach
- 34:23Key lessons and selection
related talks
36mWhat-if- Causal reasoning meets Bayesian Inference
24mAn introduction to Causal Inference with Python – making accurate estimates of cause and effect from
24mBeyond the P-Value: A Data Scientist’s Guide to Tier-1 Feature Launches
31mPython and the Holy Grail of Causal Inference
31mA Gentle Introduction To Causal Inference
33mHidden Biases In Data: Moving Beyond Black Box Models
01h 12mIntroduction to Causal Inference
37mAI/ML Strategies & Techniques for a Post Pandemic World