CAMP in the Odyssey: Provably Robust Reinforcement Learning with Certified...

on YouTube ↗13m

" summary

Introduces CAMP, a training paradigm that maximizes the provable certified radius by optimizing a surrogate loss derived from per-step local certified radii tied to the Q value gap and uses policy imitation to avoid Q overestimation instability. Reports experiments on six environments (3 classic control, 3 Atari), claims up to twice the certified expected return over Gaussian-augmented and NoisyNet baselines, and shows stronger empirical robustness under PGD and AutoAttack. Code at the linked GitHub.

AI-written summary · about

" chapters (6)

1. 0:00CAMP introduction
2. 1:17DRL robustness background
3. 3:27Research question and CAM loss
4. 5:38Policy imitation for CAM loss
5. 8:10CAMP experiments
6. 11:28Summary of contributions

" related talks

• 28mDeep Reinforcement Learning: theory, intuition, codeMaxim Lapan · PyData Amsterdam 2017
• 34mOffline Reinforcement Learning and Model-Based OptimizationSergey Levine · Reinforcement Learning from Batch Data and Simulation
• 56mA Tutorial on Reinforcement Learning IIEmma Brunskill · Foundations of Machine Learning Boot Camp 2017
• 42mReinforcement Learning using Generative Models for Continuous State and Action Space SystemsRahul Jain · Reinforcement Learning from Batch Data and Simulation
• 52mReinforcement Learning for Autonomous Cyber DefenseKrystal Jackson, Andrew Lohn · RSA Conference 2023
• 18mReward Augmentation in Reinforcement Learning for Testing Distributed SystemsRupak Majumdar, Srinidhi Nagendra, Constantin Enea, Andrea Borgarelli · OOPSLA 2024
• 48mPractical Model-based Algorithms for Reinforcement Learning and Imitation Learning, with...Tengyu Ma · Frontiers of Deep Learning 2019
• 15mMulti-Agent Reinforcement Learning In Stochastic Games: From Alphago To Robust ControlKaiqing Zhang · Meet the Fellows Welcome Event