A/B tests: Understanding user behaviour
summary
Compares a local random-assignment approach with a remote rollout via Firebase Remote Config, using an A/B example: Variant A gives one new challenge 24h after the previous, Variant B gives three new challenges each day. Implements an experiment config with a local adapter / Firebase adapter and publishes it as a package. Finds Variant B drives about a 7% increase in level play, recommends a debug menu for toggling values, testing one feature at a time, and sizing samples to detect small effects.
AI-written summary · about
related talks
34mA/B Testing You'll Like
27mHow to A/B test with confidence
28mThe Hitchhiker's Guide to Advanced A/B Testing Techniques
15mA/B Testing: Test Your Own Hypotheses & Prepare to be Wrong
24mThe Scientific Method of Experimentation
29mBalancing Personalization & A/B Testing in a Customized World
26mAnti-Patterns in A/B Testing - or How does AB Testing Scale?
19mData-driven optimisation for in-app purchases and gameplay