Emergent Alignment via Competition
summary
Claim: competition among multiple misaligned providers can yield outcomes as good as a perfectly aligned model when the user utility lies in the convex hull of provider utilities — a condition labeled market alignment. The setting is formalized as multi-sender Bayesian persuasion with providers committing conversation rules; a short Nash argument shows every equilibrium gives the user at least OPT (the aligned benchmark). With heterogeneous users or when types can be misreported, the guarantee breaks and stronger assumptions are required.
AI-written summary · about
chapters (12)
- 0:00Emergent alignment intro
- 1:24Defining AI alignment
- 3:54Doctor-AI interaction example
- 8:12LLM training as game
- 12:48Multi-sender Bayesian persuasion
- 24:48Modeling assumptions
- 31:02Market alignment proof
- 35:41Extensions to interactions
- 36:21Multiple user types
- 38:07Lying users break alignment
- 42:28Takeaways and policy
- 47:01Q&A
related talks
13mHarmonising LLM Agents for Real-World Market Intelligence:[...]
45mOn Spurious Associations and LLM Alignment
45mAGT and Machine Learning
22mGood vs Evil: Chosing the Right Moral Compass for you LLM
31mDecision theoretic foundations for human-AI collaboration
01h 08mLanguage Model Alignment: Theory & Algorithms
01h 29mLLM Safety, Alignment, and Generalization
38mSequential Information Design: Markov Persuasion Process and Its Efficient Reinforcement Learning