Optimizing your A/B testing roadmap
summary
Three practical case studies with numbers and build details. Duolingo runs over 3,000 experiments with 150–200 live at once, uses an internal training flow so anyone can run tests, and credits iterative testing for raising D1 retention from 13% to 55% while gating work with a one-pager. Dailymotion builds its own A/B testing framework, spends six months and five tests to validate a new recommendation engine and measures a north‑star premium views metric. Monzo hits limits with mutually exclusive segments, a bug that collapsed controls, and a rethink of testing cadence.
AI-written summary · about
chapters (11)
- 0:00Introduction to A/B Testing
- 0:31Duolingo's Testing Philosophy
- 3:25Prioritization and Processes
- 6:50Scaling and Empowering Teams
- 8:54Principles of Successful Testing
- 10:13Dailymotion Testing Evolution
- 13:26Product Decision Frameworks
- 17:38Major Releases and Takeaways
- 19:43Monzo's High-Tempo Testing
- 23:24Testing Constraints and Failures
- 25:16Pivotal Learnings for Monzo
related talks
52mBuilding an A/B Testing Framework that works for Developers
28mExperimenting your way to a better product
01h 15mExperimentation at scale: Fuel your application’s growth through A/B testing
36mA/B Testing in Production
34mA/B Testing You'll Like
36mA/B Testing in Production
46mAB Test Bloody Everything
05mUsing A B testing to rebuild the UI used by 1 5 million users a day