General Game-Theoretic Multiagent Reinforcement Learning
summary
Extends PSRO/Double Oracle to compute correlated equilibria by replacing the meta-solver with a maximum Gini-correlated equilibrium quadratic program, and shows empirically on small domains (TradeComm, Share, a traffic-light toy) that this yields better exploration and often reaches high-welfare solutions compared to standard meta-solvers. Shows that CFR doesn't reach EFCCE in general via a counterexample and analyzes deviation types for sequential games, points to Extensive-Form Regret Minimization variants and proposes combining no-regret population training with best-response populations as an open, practical path forward.
AI-written summary · about
chapters (22)
- 0:00Introduction
- 1:00Motivation
- 4:26GameTheoretic RL
- 6:48DoubleOracle
- 14:56Dylan
- 15:11Regret minimization
- 17:14Counterfactual regret
- 18:39Looking back
- 19:30A specialized environment
- 20:43Correlated equilibrium
- 22:35Causal equilibria
- 26:06Our paper
- 26:45What are the problems
- 28:08How do we address these problems
- 28:25No Regret
- 29:32Main Open Question
- 29:46Questions
- 30:39Other algorithms
- 33:06Traffic lights
- 37:26Where do you go
- 37:49Correlation Equilibrium
- 38:50Conclusion
related talks
47mRebel: Combining Deep Reinforcement Learning and Search for Imperfect-Information Games
19mTraining Agentic Reasoners
30mReinforcement Learning - ChatGPT, Playing Games & More
45mAGT and Machine Learning
41mSpotlight Talks
38mRetrospective on multi-agent learning as a path to intelligence
31mFrom Single AI to Multi AI Systems
42mRethinking AI Alignment and Evaluation under Heterogeneous Preferences