Strong Truthfulness in Multi-signal Peer Prediction with Overlapping Tasks
summary
Paper designs score-matrix mechanisms for peer prediction with overlapping tasks and multi-valued signals, proves the output-agreement generalization is strictly proper and strongly truthful on categorical domains, and introduces a zero-one mechanism that is informed-truthful for general domains. The ABCD mechanism yields strong truthfulness among symmetric strategies (not proper) but shows all equilibria in large systems pay less than truthful play. Analysis uses a Delta matrix of joint minus product marginals and tests ~104 questions across 17 MOOCs (~325k responses).
AI-written summary · about
chapters (19)
- 0:00Intro
- 2:57Information Elicitation Without Verification
- 4:00Simple Output agreement
- 7:36The Peer Prediction Method
- 9:161/Prior, Shadowing Method
- 10:47Mechanism Desiderata
- 14:55What do we know?
- 17:34Experimental evidence
- 18:47Our Contributions: Robust, Multi-Signal Methods
- 20:46Basic Set-up
- 29:30Solution Concepts
- 33:19Family of Mechanisms
- 36:17Analysis: Expected Payment
- 41:39Deterministic Strategies
- 41:56Simplified analysis
- 43:51The 01-mechanism
- 44:26The ABCD-mechanism
- 46:40Delta matrices: MOOC Data
- 47:29Discussion
related talks
35mMore Dominantly Truthful Multi-task Peer Prediction with a Finite Number of Tasks
44mClearing Matching Markets Efficiently: Informative signals and Match Recommendations
29mClearing Matching Markets Efficiently: Informative Signals and Match Recommendations
50mDiversity in Matching Markets
59mMechanism Design with Limited Commitment
32mDynamic Matching Models
20mWhat Can Cryptography Do For Decentralized Mechanism Design?
52mDashboard Mechanisms for Online Marketplaces