High-Dimensional Statistics II
summary
General recipe: M‑estimators with convex loss plus a decomposable regularizer yield concrete error rates once restricted strong convexity holds and the dual‑norm score is controlled. Choose lambda ∝ sqrt(log d / n) (union‑bound style) and the oracle inequality gives L2 error ≲ Rq · (log d / n)^{(1−Q)/2} for LQ balls (Q∈[0,1]); Q=0 recovers parametric 1/n scaling, Q=1 gives 1/√n. Applications: logistic and GLMs, group L1, nuclear‑norm low‑rank recovery, matrix completion and robust low‑rank+sparse decomposition; these rates are minimax optimal.
AI-written summary · about
chapters (12)
- 0:00Introduction to M-estimators
- 0:39Oracle inequalities and sparsity
- 3:23Analyzing error bounds
- 5:21Thresholding and optimization
- 7:07Algebraic derivation of rates
- 12:28Oracle inequalities in practice
- 17:13Minimax risk and lower bounds
- 22:12Information-theoretic techniques
- 32:23Structured estimation models
- 45:30General principles for M-estimation
- 1:02:56Duality and score functions
- 1:12:43Application to low-rank matrices
related talks
41mTransfer learning via local convergence rates of the nonparametric least squares estimator
29mImplicit Regularization in Nonconvex Statistical Estimation
53mOptimality in Locally Private Estimation and Learning
48mComputational-Statistical Tradeoffs in Robust High-Dimensional Estimation
46mCan Non-Convex Optimization be Robust?
01h 23mImplicit Regularization II
49mFast Algorithms for Regression Problems
01h 01mRecent Advances in Algorithmic Heavy-Tailed Statistics