Reinforcement Learning using Generative Models for Continuous State and Action Space Systems
summary
Algorithm RandPaul uses a generative simulator, empirical Bellman updates and an actor‑critic with randomized shallow nets (only last layer learned) and sample averages N_Q, N_pi, M and J_Q/J_pi. Theory gives nonasymptotic L1 error bounds with sample complexity ~epsilon^{-2} and log(1/delta) dependence. Implementation runs on a Minitaur PyBullet quadruped, beats random search, matches PPO in reward but requires 10–12 hours more training, while DDPG underperforms. L2 function fitting produces the deployable Q and policy.
AI-written summary · about
chapters (13)
- 0:00Introduction
- 2:21Autonomous Systems
- 3:55Model Free Approaches
- 5:18Reinforcement Learning
- 6:39Optimal Value Function
- 9:09Continuous State Space
- 11:20Actor Critic Architecture
- 13:53Neural Networks
- 16:21Policy Evaluation
- 22:52Theorem
- 24:27Does it work
- 27:50Conclusion
- 30:56Questions
related talks
31mLearning to Control Unknown Systems
11mData Efficient Methods for Reinforcement Learning
56mA Tutorial on Reinforcement Learning II
28mDeep Reinforcement Learning: theory, intuition, code
01h 01mA Tutorial on Reinforcement Learning I
01h 17mTraining intelligent game agents using deep reinforcement learning
01h 11mFoundations of RL
15mMulti-Agent Reinforcement Learning In Stochastic Games: From Alphago To Robust Control