Practical Multi Armed Bandits
summary
Concrete takeaways: implemented systems combine epsilon-greedy with Thompson sampling, compute updates in NumPy and SciPy, and store minimal state for fast online updates. Reward design is the core — weighted, immediate signals (clicks, pings, synced objects) drive the policy. Change detection uses anomaly resets or parallel "naive" learners; a production deployment saved about $600/month by preferring low-latency peers. Examples include contextual bandits, UCB variants, and timebound sprints trained via quick pairwise games.
AI-written summary · about
chapters (11)
- 0:00Introduction
- 1:04Multi-armed bandit problem
- 3:10Exploration vs exploitation
- 6:44Bandits in A/B testing
- 8:51Bandits and reinforcement learning
- 13:53Handling change in algorithms
- 17:29Implementing bandits in practice
- 23:50Server synchronization example
- 27:21News aggregator ranking
- 28:47Sprint prioritization
- 30:11Closing and Q&A
related talks
21mLeveraging Multi-Armed Bandit Algorithms for Dynamic Decision Making
17mByron Galbraith, Chief Data Scientist, Talla, NYC 2017
32mMulti-Player Multi-Armed Bandit: Can We Still Collaborate at Homes Without "Zoom"?
01h 01mStochastic Bandits: Foundations and Current Perspectives
Bandits and Agents: How to Incentivize Exploration?
32mMulti-Armed Bandits in Recommendations
01h 21mAdversarial Bandits: Theory and Algorithms
48mAdaptivity and Confounding in Multi-armed Bandit Experiments