Getting Started with Multi-variant Testing
summary
Team reintroduces user testing by wiring LaunchDarkly feature flags into the registration flow as a quick front-end proof of concept, serving full JSON banner variations per venture, tracking registration completion and events. Experiment reaches 97% posterior probability for the image variant, showing ~1.5% absolute lift in conversions (from a 24% base funnel, a ~56% relative uplift to the affected step). Workflow uses A/B and multivariate testing and plans to move to multi-armed bandit allocation and cleaner server-side integrations.
AI-written summary · about
chapters (11)
- 0:00Introduction
- 2:50Multi-variant testing for secondary products
- 5:40Measuring small product changes
- 8:35Christmas backlog for experimentation
- 9:17First experiment with LaunchDarkly
- 12:55Analyzing experiment results
- 14:21Unexpected outcome and decisions
- 16:29Success metrics and impact
- 17:11Key takeaways
- 20:47Expanding experimentation
- 21:31Q&A
related talks
36mA/B Testing in Production
36mA/B Testing in Production
28mOptimizing your A/B testing roadmap
52mBuilding an A/B Testing Framework that works for Developers
46mAB Test Bloody Everything
41mA/B Testing with Style
48mHow to use experimentation in the back-end
13mPersonalization and A/B Testing: How to create better experiences for users