Confident Off-policy Evaluation and Selection through Self-Normalized Importance Weighting
summary
Derives non-asymptotic, high-probability, data-dependent lower bounds for self-normalized importance weighting in contextual bandits, using a semi-empirical Efron–Stein variance proxy that is computable given a known stationary behavior policy; avoids weight truncation, yields multiplicative bias that shrinks when weight distributions are not skewed, and captures an effective-sample-size regime from 1/n to constant. Proves concentration via a self-normalized bounded-difference argument and shows in synthetic and recommender-style experiments tighter, non-vacuous confidence bounds and improved policy selection under limited overlap.
AI-written summary · about
chapters (9)
- 0:00Introduction to the topic
- 1:24Contextual bandit model
- 4:39Self-normalized estimation
- 7:21Concentration analysis
- 12:35Advanced concentration methods
- 18:01Value bounds and experiments
- 23:31Policy selection in practice
- 27:11Proof techniques and summary
- 32:03Q&A and concluding remarks
related talks
53mOff-policy Policy Optimization
09mTowards Credible and Effective Data-Driven Decision-Making
41mKeynote: Personalization, Bandits, and Causal inference
22mPersonalizing Explainable Recommendations with Multi-objective Contextual Bandits
29mPure Exploration Problems
01h 01mStochastic Bandits: Foundations and Current Perspectives
35mEstimating the Value of Personalization
32mDesigning Adaptive Experiments For Policy Learning And Inference