Make Better Decisions: Experiment!

on YouTube ↗20m

" summary

Etsy runs platform experiments that prove or disprove precise hypotheses: infinite scroll reduces items viewed, purchases and favorites, while simple variants like 80 items per page or loading results earlier show no effect, saving development work. Apps use traffic splitters (Optimize, Lean Firebase, Bugsnag coming) and feature flags so changes toggle without redeploy. Bucket users at reveal time to avoid skew, wait for significance (commonly 5% alpha, 80% power) and avoid peeking; a 50/50 split nav test increased favorites and rolled out.

AI-written summary · about

" chapters (9)

1. 0:00Introduction to experimentation
2. 1:05Why experiment matters
3. 2:51Infinite scroll case study
4. 5:59How to run experiments
5. 8:07App-specific experimentation tips
6. 11:17Statistical gotchas explained
7. 14:07Don’t peek at early data
8. 17:14Beyond button color tests
9. 19:43Summary and call to action