Successfully Applying Bayesian Statistics to A/B Testing in Your Business
summary
The talk examines using Bayesian statistics—especially hierarchical Bayesian models implemented with PyMC—to improve A/B testing in business. It compares frequentist A/B calculations (normal vs beta assumptions, multiple comparisons, and biased stopping times) to Bayesian approaches, evaluates whether those theoretical issues matter in practice, and presents a practical stopping rule for when to end an insignificant test and how that rule relates to the theoretical concerns.
AI-written summary · about
chapters (7)
- 0:00Bayesian A/B testing intro
- 3:56Bayesian vs traditional
- 11:17Implementing Bayesian testing
- 19:07Choosing priors and results
- 26:39When to stop a test
- 31:15Real-world examples and pitfalls
- 37:42Key takeaways
related talks
38mUsable A/B testing – A Bayesian approach
25mScaling Bayesian Experimentation
42mSolving Real-World Business Problems with Bayesian Modeling
41mA/B Testing with Style
29mHow to use Data Science Superpowers in real life, a Bayesian perspective
01h 24mBayesianism and Survival Analysis
41mBayesian methods in data analysis, an introduction
34mHypothesis to Impact: Mastering the Art and Science of Controlled Experiments