A Theoretical Framework of Convolutional Kernels on Image Datasets
summary
Characterizes the RKHS of single‑layer convolution + pooling + downsampling and analyzes kernel ridge regression and random‑feature limits under isotropic data on the sphere, proving that convolution breaks the curse of dimensionality by restricting the RKHS to Q‑local functions, global average pooling enforces translation invariance, and local pooling biases toward low‑frequency functions. Main quantitative claim: fitting degree‑L targets drops sample complexity from about D^L to D^{L−1} (or replaces D by q); example: D=30, q=10, degree‑3 needs ~20,000 (FC) vs ~930 (cyclic) vs ~100 (locally connected + invariant).
AI-written summary · about
chapters (8)
- 0:00Start
- 6:20Generative Model
- 14:30Why Is It Called an Invariant Neural Network
- 30:07Experiments
- 34:02Locality
- 39:12Eigen Functions
- 49:43Theoretical Prediction
- 55:34Open Questions
related talks
33mThe Power and Limitations of Kernel Learning
12mDeep Neural Tangent Kernel and Laplace Kernel Have the Same RKHS
01h 22mGeneralization IV
51mTwo Concise Representations for High-Dimensional Visual Data
30mKernel Methods For Causal Inference
01h 04mLearning Staircases
01h 10mSketching for Linear Algebra: Basics of Dimensionality Reduction and CountSketch II
44mConcentration Inequalites in Learning with Kernels