Asympirical Analysis: Theory Informs Practice
summary
Proposes asympirical analysis: use asymptotic formulas to extrapolate tuning parameters found on a small randomized subsample back to the full-sample nonparametric fit. Implements simple randomized linear algebra subsampling and applies generalized cross‑validation on the subsample, then scales the GCV-suggested smoothing lambdas via the theory for tensor-product smoothing splines in a tensor-product RKHS. Tests show b much smaller than n (repeat subsamples ~5), recovers full-sample GCV when n≈40k and works on n≈1M, runs far faster and beats a common “skip/miser” initializer.
AI-written summary · about
chapters (15)
- 0:00Start
- 0:39Machine Learning Paradigm
- 2:19Classic Statistics Paradigm
- 3:41Big Data Era
- 5:49Nonparametric Model (Gaussian Process)
- 7:01Reproducing Kernel Hilbert Space
- 9:07Data Driven Smoothing Parameter Selection
- 9:47Asymptotic Theory
- 12:00Theory Informs Practice
- 15:17Rule of Thumb
- 16:31Simulation
- 18:37Setting 1: Additive Model, 3 main effects
- 19:24Setting 3: 8 main effects with 28 interactions
- 20:26Beijing PM 2.5
- 21:42Molecular Dynamics
related talks
33mRandomized Least Squares Optimization and its Incredible Utility for Large-Scale Tensor Decomp...
01h 03mSampling for Linear Algebra, Statistics, and Optimization I
01h 02mTheory and Big Data
01h 27mPast, Present and Future of Randomized Numerical Linear Algebra I
01h 02mSampling for Linear Algebra, Statistics, and Optimization II
53mSketching and Projecting
01h 07mStochastic Second Order Optimization Methods I
33mStochastic Optimization for Big Data Machine Learning Problems