Learning Automata as Building Blocks for MARL
summary
Learning Automata using the Reward-in-Action update directly tune action probabilities and have a built-in exploration that, by Narendra and Wheeler’s theorem, converge to pure Nash equilibria in n‑person non‑zero‑sum games. They prove robust performance with asynchronous, delayed feedback (queueing/chemical plant examples) and produce periodic policies that alternate between pure equilibria which beat mixed strategies when job chunks are large. Implementations use per‑state automata (Markov games) and simple function approximators; open challenge is integrating the update scheme with deep neural function approximation.
AI-written summary · about
chapters (10)
- 0:00Intro
- 6:36Commitment Sequences
- 10:05Learning Automata
- 13:38Learning Curves
- 14:36Nash Equilibrium
- 16:13Nash Equilibrium in Action
- 18:00Pure Nash Equilibrium
- 19:35Examples
- 22:32Continuous Action
- 25:16Conclusion
related talks
25mDesigning and Building Custom Reinforcement Learning Environments for Fine-tuning LLMs
36mOn the Road to Artificial General Intelligence
01h 29mLLM Safety, Alignment, and Generalization
45mAGT and Machine Learning
15mAgentic Reasonings with Reinforcement Learning
39mLet LLMs Wander: Engineering RL Environments
15mMachine learning with matches and confetti
38mRetrospective on multi-agent learning as a path to intelligence