A Bluffer's Guide to Dimension Reduction
summary
The claim is that almost everything useful about dimension reduction reduces to two concrete mechanisms: matrix factorization — build a reconstruction loss (PCA, NMF, Poisson/LDA variants) and vary constraints to get clustering or topic models — and neighbor graphs — build KNN kernels, graph Laplacians, then embed with eigenvectors or a force-directed layout (t-SNE, UMAP). The talk shows how those constructions are implemented, why kernel choice and constraints matter, and where computational and interpretability tradeoffs arise.
AI-written summary · about
chapters (19)
- 0:00Dimension reduction intro
- 1:47Matrix factorization core
- 3:30Factorization examples & PCA
- 6:43Sparse PCA & k-means
- 8:50Non-negative matrix factorization
- 10:57Exponential family PCA
- 13:48PCA as matrix factorization
- 15:11Probabilistic factorization techniques
- 17:39LDA as factorization
- 19:05Neighbor graphs overview
- 21:11Building neighbor graphs
- 22:56Spectral embedding & Laplacian
- 24:23t-SNE intuition
- 26:07Graph layout basics
- 27:51UMAP core intuition
- 29:13Summary of techniques
- 30:18Q&A: t-SNE & UMAP
- 33:11Computational complexity
- 34:57Q&A: graph optimization
related talks
34mDimensionally reducing data squeezing out the good stuff
40mA Practical Guide to Dimensionality Reduction Techniques
01h 07mA Few Simple Average-Case Reduction Techniques And Their Surprising Effectiveness
40mTopological Techniques for Unsupervised Learning
21mA Visual Interaction Framework for Dimensionality Reduction Based Data Exploration
01h 03mRecent Progress in High-Dimensional Learning
38mManifold Learning and Dimensionality Reduction for Data Visualization
01h 02mTheory and Big Data