On Rewards in Multi-Agent Systems
summary
Empirical claim: adding agent-to-agent rewards mitigates Tragedy of the Commons. Experiments run in the harvest domain with 10 agents, apples that only regrow above a density threshold, a tagging beam and a new gifting beam; agents train with DQN with epsilon-greedy. Three gifting rules (zero-sum, fixed-budget, replenishable) are tested and zero-sum gifting avoids the mid-training collapse and decays to equilibrium more slowly; equality (Gini) ends up similar. Separately, reward-design is solved via a polynomial-time LP over occupancy measures for several task formalisms, but some preferences are not realizable by Markov rewards and full environment knowledge is assumed.
AI-written summary · about
chapters (14)
- 0:00Intro
- 0:30The Importance of Multi-Agent Systems for Reinforcement Learning
- 1:48Gifting in Multi-Agent Reinforcement Learning
- 3:19Why Gifting?
- 4:37Gifting Mechanisms
- 6:37Case Study: Harvest Domain (Perolat et al, 2019)
- 8:17Case Study: Performance
- 11:01Case Study: Tagging Behavior
- 12:07Case Study: Gifting Behavior
- 13:02Summary so far
- 15:52Examples of Task Types
- 17:13Task Realization
- 20:57Reward Design
- 22:13Discussion
related talks
19mTraining Agentic Reasoners
42mRethinking AI Alignment and Evaluation under Heterogeneous Preferences
50mScalable Innovation: Exploring Multi-Agent Systems
26mAI Agents and the Future of Human-AI Collaborative Intelligence
36mOn the Road to Artificial General Intelligence
31mFrom Single AI to Multi AI Systems
38mRetrospective on multi-agent learning as a path to intelligence
32mMulti-Player Multi-Armed Bandit: Can We Still Collaborate at Homes Without "Zoom"?