Dimensionally reducing data squeezing out the good stuff
summary
Explains which algorithms actually reduce dimensions: k‑means as distance clustering, PCA via variance (example directions explaining ~30 vs ~1.4 variance) and t‑SNE for nonlinear structure, implemented in Python with a scikit‑learn demo on a Netflix ratings sparse matrix from GitHub. Shows preprocessing choices (center nonzero ratings, truncate for RAM), extracts polarizing films that drive variance, and notes what works (fast linear algebra exploration) and what fails (PCA on sparse/binary data).
AI-written summary · about
chapters (14)
- 0:00Data dimensionality intro
- 1:06Data structure basics
- 2:27Data vs raw observations
- 3:10Data space concept
- 6:01K-means clustering
- 8:50K-means limitations
- 9:11PCA and variance
- 13:48PCA math explained
- 15:15PCA applications
- 17:05Netflix PCA demo
- 23:14t-SNE intuition
- 25:19Data scientist steps
- 26:08PCA & t-SNE Q&A
- 34:20Summary
related talks
37mA Bluffer's Guide to Dimension Reduction
40mA Practical Guide to Dimensionality Reduction Techniques
01h 02mTheory and Big Data
38mManifold Learning and Dimensionality Reduction for Data Visualization
40mTopological Techniques for Unsupervised Learning
01h 07mA Few Simple Average-Case Reduction Techniques And Their Surprising Effectiveness
40mTools and Tricks from a Pragmatic Data Scientist
02h 23mEffective Data Visualization