Multi-Agent Reinforcement Learning Towards Zero-Shot Communication

on YouTube ↗58m

" summary

Paper introduces zero-shot communication and an algorithm (QED) that trains many sender/receiver pairs under a Zipfian goal prior and action energy cost, then iteratively infers action equivalence mappings by minimizing KL between one-hot actions and learned transforms. Core claim: maximizing mutual information between goals and action-equivalence classes yields protocols that cross-play with novel partners (self-play cross-play ~45% vs QED near self-play), at the cost of higher energy use. Continuous-trajectory tests show separable energy Gaussians but heavy sample complexity and only ~10% runs reach global optima.

AI-written summary · about

" chapters (23)

1. 0:00Intro
2. 0:57WHY Cooperative Multi-Agent Systems?
3. 4:32Background: Learning a Communication Protocol
4. 4:58Background: Referential Games for Protocol Learning
5. 6:05Related Work - Learning Multi-Agent Communication
6. 6:41Problem formalism (Fully Cooperative)
7. 7:39Background: (Cooperative) Multi-Agent Reinforcement Learning
8. 8:04Background: Multiagent Training Regimes
9. 8:41Problem Setting: Zero-Shot Coordination
10. 10:37Problem with Training using Self-Play
11. 13:02Inducing Bias: Priors from Real-World Settings . Cost of Physical Energy Exertion
12. 14:24What about Symmetries in Realistic Communication?
13. 15:23How does Symmetry impact the Optimization Problem?
14. 19:22Baseline Training using SP
15. 19:52Method: Exploit Equivalence Mappings using OP
16. 25:47Method: Learning Algorithm inferring Equivalence Mappings
17. 29:17GED Method Inferring Equivalence Mappings (Symmetries)
18. 29:55Playing the Signaling Game with Symmetries
19. 33:47What about the Continuous Channel Setting?
20. 35:16Method: SP Protocol Training
21. 36:03Method: SP Protocol + Third-Party Receiver Training
22. 39:30Recap of Zero-Shot Communication Work Motivating Goat Emergence of Communication Protocols that Generalize
23. 40:10Open Challenges for ZS Communication

" related talks

• 49mInterpreting Emergent CommunicationYonatan Belinkov · Decoding Communication in Nonhuman Species III (2024)
• 47mRebel: Combining Deep Reinforcement Learning and Search for Imperfect-Information GamesNoam Brown · Adversarial Approaches in Machine Learning 2022
• 32mMulti-Player Multi-Armed Bandit: Can We Still Collaborate at Homes Without "Zoom"?Yuanzhi Li · Mathematics of Online Decision Making 2020
• 38mRetrospective on multi-agent learning as a path to intelligenceIgor Mordatch · Understanding Higher-Level Intelligence from AI, Psychology, and Neuroscience Perspectives
• 41mSpotlight TalksRashida Hakim, Chhavi Yadav · Agency in Collaborative Learning 2026
• 01h 03mIEEE CoG 2019 - Industry Day - David Silver (DeepMind)David Silver · IEEE Conference on Games 2019
• 14mUnconference: Self-optimizing deep research agentsJakub Zavrel · OpenSearchCon North America 2025
• 21mUsing Deep Reinforcement Learning for Dialogue SystemsHarm van Seijen · MLconf SF 2016