A Complex Picture of Multi-task Learning
summary
Multitask learning with N source datasets each of size n yields a mixed picture: an oracle vs adaptive gap where an oracle that knows which M sources are close to the target can get aggregate-sample rates much faster than using target data alone, but no adaptive procedure (without side information) can guarantee that improvement because identifying the good sources incurs a combinatorial multiple-hypothesis-testing cost. Positive results require extra info such as ranking knowledge suffices or low-noise margins; in regression, linear shared-subspace recovery uses averaged outer products and sine-theta subspace estimation for fast rates, while nonlinear RKHS cases force overfit regressions (regularization bias) and need stronger smoothness.
AI-written summary · about
chapters (16)
- 0:00Multitask learning intro
- 1:26Goals and assumptions
- 6:24Key questions on gains
- 8:20Unstructured vs structured
- 11:17Unstructured classification setup
- 13:42Challenges in gains
- 15:32Source-target relationships
- 21:20Minimax rates and oracle
- 23:33Negative results on adaptivity
- 27:58Why practical algorithms fail
- 31:55Open questions and tightness
- 32:16Structured regression overview
- 34:10Linear structured regression intuition
- 37:55Nonlinear structured regression challenges
- 42:07Summary of mixed results
- 43:13Q&A on limitations
related talks
38mMulti-objective Learning: An Algorithmic Toolbox for Optimal Predictions Anytime Anywhere!
06mCross-Dataset Activity Recognition via Adaptive Spatial-Temporal Transfer Learning
28mBuilding one multitask model to rule them all
45mMulticalibration, Universal Adaptability and Causality
27mLearning and Strongly Truthful Multi-task Setting Peer Prediction: A Variational Approach
31mMulti-Task Optimal Experiment Design
40mOmnipredicting Single-Index Models with Multi-Index Models
01h 11mA Theory of Multi-objective Machine Learning