A Tutorial on Reinforcement Learning I

on YouTube ↗01h 01m

" summary

Concise tutorial on reinforcement learning that defines MDPs, contrasts model‑based vs model‑free (tabular Bellman backups and Q‑learning), and treats exploration with concrete algorithms: epsilon‑greedy, optimism via UCB, and Bayesian Thompson sampling with posterior updates. Theory gives PAC‑MDP bounds and a startling sample example—tiny 10‑state problems needing ~10^12 samples in early bounds, later tightened toward ~10^6—and explains why episodic structure and refined “knownness” reduce those rates.

AI-written summary · about

" chapters (15)

1. 0:00Introduction to RL
2. 1:14RL Applications
3. 3:02RL vs Machine Learning
4. 5:48Markov Decision Processes
5. 10:45Model-based Reinforcement
6. 12:16Q-learning
7. 18:51Exploration vs Exploitation
8. 22:14Epsilon-greedy Strategy
9. 24:00Optimism under Uncertainty
10. 27:38Bayesian Methods
11. 31:43Optimal Exploration
12. 38:34Evaluating RL Algorithms
13. 43:03Asymptotic Optimality
14. 45:20PAC Reinforcement Learning
15. 54:22Tighter PAC Bounds

" related talks

• 01h 05mFoundations of RLRayadurgam Srikant · Data-Driven Decision Processes Boot Camp 2022
• 42mReinforcement Learning using Generative Models for Continuous State and Action Space SystemsRahul Jain · Reinforcement Learning from Batch Data and Simulation
• 31mLearning to Control Unknown SystemsRahul Jain · Societal Networks 2018
• 44mReinforcement learning: A gentle introductionUwe Friedrichsen · Craft Conference 2019
• 48mPractical Model-based Algorithms for Reinforcement Learning and Imitation Learning, with...Tengyu Ma · Frontiers of Deep Learning 2019
• 52mReinforcement Learning in Recommender Systems: Some ChallengesCraig Boutilier · Emerging Challenges in Deep Learning (Simons Institute workshop, 2019)
• 31mOn the Complexity of Learning Good Policies With and Without RewardsEmilie Kaufmann · Mathematics of Online Decision Making 2020
• 01h 11mFoundations of RLRayadurgam Srikant · Data-Driven Decision Processes Boot Camp 2022