Independent Component Analysis: From Theory to Practice and Back
summary
Problem: recover unknown linear mixing A from samples X = A S where S has independent coordinates and at most one Gaussian. Algorithm: make samples isotropic, reweight by complex Fourier factors e^{i u^T x} and do PCA on the reweighted covariance (Fourier PCA); algebra shows the Hessian equals A · diag(ψ''_i(A^T u)) · A^T so eigenvectors reveal columns of A when cumulants differ from Gaussian. Tools: tensor power iteration for orthogonal case and polynomial anti-concentration bounds on eigenvalue gaps. Claim: polynomial sample/time when the first nonzero cumulant order K is constant; a recursive spectral partitioning heuristic cuts samples to near-linear under a fourth‑moment assumption.
AI-written summary · about
chapters (12)
- 0:00Introduction to ICA
- 0:28ICA model definition
- 0:47Independent Component Analysis
- 6:40Fourth moment algorithm
- 15:13Tensor power iteration
- 20:37General ICA solution
- 25:37Reweighted PCA algorithm
- 29:15Analysis of new algorithm
- 35:51Polynomial anti-concentration
- 38:28Underdetermined ICA
- 40:29Recursive ICA and gaps
- 45:20Open problems and Q&A
related talks
01h 31mUnsupervised Learning
29mNon-Gaussian Component Analysis: Statistical Query Hardness and its Applications
39mMemory Limited, Streaming PCA
29mApproximating Spherical Gaussian Mixtures
49mApproximate Message Passing Algorithms For Orthogonally Invariant Models
52mLearning Mixture of Gaussians using Moment Tensor
01h 03mSampling for Linear Algebra, Statistics, and Optimization I
35mLearning General Gaussian Mixtures With Efficient Score Matching