Multiagent Reinforcement Learning: Rollout and Policy Iteration
summary
A new multiagent policy-iteration method tests one agent at a time instead of all joint actions, using dynamic programming plus simulation and neural value and policy networks for approximation. It reduces exponential branching to linear (e.g., 5^M comparisons → 5×M), keeps the rollout improvement property and converges finitely for the exact case to an agent-wise Nash point. In a grid capture case the greedy base policy takes 85 steps, standard rollout 34 steps, while agent-by-agent rollout matches performance with far fewer comparisons; POMCP/DASPt and MAD-PG fare worse in the experiments.
AI-written summary · about
chapters (9)
- 0:00Sources
- 1:49Multiagent Problems - A Very Old (19608) and Well-Researched Field
- 12:37For this Talk we Focus on Finite-State Intinite Horizon Problems
- 15:06Policy Iteration (PI) Algorithm
- 18:00Outline of Our Approach for Multiagent Problems
- 22:22Underlying Theory: Trading off Control and State Complexity (NDP book, 1996)
- 29:06Comparing Standard with Multiagent Rollout/Policy Iteration
- 31:25Approximate Policy Iteration with Agent-by-Agent Policy Improvement
- 33:23Concluding Remarks
related talks
01h 17mTraining intelligent game agents using deep reinforcement learning
15mSequential Decision Making Problems in Multi-Agent Environments
21mScalable agent-based models for optimized policy design: applications to the ec...
15mMulti-Agent Reinforcement Learning In Stochastic Games: From Alphago To Robust Control
14mHuman-in-the-Loop Reinforcement Learning
11mUsing Deep Reinforcement Learning to Coordinate Multi-Modal Journey Planning with Limited ...
15mRollArt: Disaggregated Multi-Task Agentic RL Training at Scale
47mHow agentic AI solves complex scheduling problems