On Learning Distributions from their Samples
summary
Characterizes minimax risk for several losses with concrete estimators and sharp constants: for relative entropy the first-order minimax term is (k-1)/(2n) and the optimal rule is a varying add-β scheme that perturbs counts (β0=1/2, β1=1, later β≈3/4). For chi-squared loss the paper gives non‑asymptotic upper and lower bounds, the leading term is (k-1)/n, the Laplace add-one estimator attains it, and the second‑order error is at most logarithmic (O(log n / n^2)). Also shows f‑divergence formula when probabilities ≥ α.
AI-written summary · about
chapters (10)
- 0:00Introduction
- 0:20Problem Statement
- 2:32Measures of Loss
- 3:51Relative Entropy
- 4:53Laplace
- 9:24Chi squared distance
- 11:49Probability distributions
- 14:01Add beta estimator
- 17:46Serendipity
- 21:41Summary
related talks
15mFundamental Limits Of Learning In Data-Driven Problems
53mOptimality in Locally Private Estimation and Learning
01h 15mAn overview of classical robust statistics and generalizations to the future
41mTransfer learning via local convergence rates of the nonparametric least squares estimator
01h 23mGeneralization II
01h 28mMulti-Distribution Learning, for Robustness, Fairness, and Collaboration
53mDesigning Robust Learners
45mTesting and Learning Distributions Under Local Information Constraints