Clustering: A Guide for the Perplexed
summary
Clustering recommendation: use HDBSCAN rather than kmeans because it outputs noise/outliers, finds variable-density clusters, and avoids forcing every point into spherical clusters. Implementation compiles to C, uses a dual-tree minimum spanning tree to achieve roughly O(n log n) behavior, and runs interactively on ~100k points or overnight on 5–10 million points. The method derives from persistent‑homology ideas and sheaf generalizations, and the available Python package performs well in practice.
AI-written summary · about
chapters (24)
- 0:00Clustering introduction and challenges
- 1:45K-means clustering algorithm
- 3:10Centroid-based clustering limits
- 4:14Affinity propagation trade-offs
- 4:37Mean shift clustering assumptions
- 5:20Spectral clustering assumptions
- 6:24Birch algorithm for scalability
- 7:10Hierarchical clustering dendrograms
- 8:36DBSCAN density-based clustering
- 9:38Evaluating clustering quality
- 11:02Practical cluster definitions
- 12:26Density-based clustering explained
- 14:52Density approximation with Leland
- 17:40Density trees for clustering
- 19:04Simplified density trees
- 21:12HDBSCAN algorithm
- 23:25Optimizing HDBSCAN indexing
- 24:52Scaling HDBSCAN to data
- 26:17Persistent density clustering
- 28:26Topological clustering approaches
- 29:28Key takeaways and recommendations
- 30:11Q&A high dimensions
- 31:16Q&A performance and scalability
- 32:41Q&A persistent homology
related talks
34mHDBSCAN, Fast Density Based Clustering, the How and the Why
26mUnleash Big Data Clustering
26mMathematical Approaches to Clustering
01h 20mClustering Data
47mAssessing the quality of a clustering
25mThe Art of Clustering: The Good, The Bad and The Beautiful
01h 00mStreaming Euclidean k-median and k-means with o(log n) Space
34mParallel Efficient Clustering with Graph-Based Approximate Nearest Neighbor Search