Tutorial: Incentives for Collaborative Learning and Data Sharing, Part II
summary
Covers practical definitions and limits of data valuation: leave-one-out needs N+1 retrainings and fails on redundancy; Shapley value solves redundancy but costs 2^N retrainings so people sample permutations or K epochs; continuous influence functions use a Hessian-inverse formula to drop cost to one training but require expensive Hessian-vector solves and break under nonconvexity. Shows a 52B‑parameter test that locates influential training texts, and flags open problems: statistical uncertainty, validation reuse, distributed protocols and incentive-aware mechanisms.
AI-written summary · about
chapters (12)
- 0:00Introduction
- 1:13Data-centric AI motivation
- 6:40Data contribution attribution challenges
- 15:33Leave-one-out to hypothesis testing
- 26:51Shapley values and axioms
- 46:09Shapley values limitations
- 51:52Approximating Shapley values
- 1:05:31Influence functions approximations
- 1:11:51Empirical data valuation methods
- 1:19:29Data selection pitfalls
- 1:22:52Data valuation and mechanism design
- 1:25:39Summary
related talks
48mIncentivizing Truthful Data Sharing in Collaborative Machine Learning and Data Marketplaces
44mWhat is Your Data Worth? Equitable Data Valuation in Machine Learning
51mEfficient Retrieval of Influential LLM Training Examples
38mTowards Privacy-Respectful Cyber-Physical Systems: Two Perspectives
01h 28mMulti-Distribution Learning, for Robustness, Fairness, and Collaboration
37mActive Teaching, Human Learning
42mRethinking AI Alignment and Evaluation under Heterogeneous Preferences
52mMechanisms for Honest Data Sharing under Competition