Personalized HeartSteps: A Reinforcement Learning Algorithm for Optimizing Physical Activity
summary
A reinforcement learning algorithm for HeartSteps V2 decides five times per day whether to send a walking suggestion using a Bayesian low-dimensional linear reward model with an informative prior from HeartSteps V1. It estimates posteriors with action-centering to protect against misspecification and nonstationarity, encodes delayed harms via a dosage variable (discounted sum of past treatments) as a proxy for future reward, and picks actions stochastically like Thompson Sampling with a forced minimum probability. It beats standard Thompson Sampling in simulation and shows promising pilot results.
AI-written summary · about
chapters (7)
- 0:00Introduction
- 0:21Background
- 1:14Explanation
- 1:45Study Design
- 2:23Challenges
- 3:59Algorithm
- 5:05Conclusion
related talks
12mDemystifying Reward Design in Reinforcement Learning for Upper Extremity Interaction: Practical G...
38mReinforcement Learning - What it is and How it Works
52mReinforcement Learning in Recommender Systems: Some Challenges
49mIncentivized Exploration
14mHuman-in-the-Loop Reinforcement Learning
21mKeynote: Fueling Innovation Engines by Creating, Collecting, and Improving Stepping...
28mShift your ML model into overdrive with AWS DeepRacer analysis tools
32mHacking Fitness