Jorge Silva, Sr. Research Statistician Developer, SAS
summary
Proposes the aligned box criterion (ABC), a scalable implementation that builds data-adaptive reference distributions by recursively forming PCA-aligned bounding boxes around clusters and re-simulating k-means to compute a gap-like statistic. Implements ABC in SAS’ High Performance Analytics (HPA / HProc HP+) and shows it favors smaller k, produces clearer global maxima than CCC or the vanilla gap, yields near state-of-the-art accuracy on Kaggle claims data, and runs with roughly linear scaling and low variance across initializations.
AI-written summary · about
chapters (8)
- 0:00Start
- 0:08K-Means Clustering
- 0:36How Many Clusters Should Be in a Data Set
- 2:10Aligned Box Criterion
- 6:14The Elbow Method
- 9:13Gap Statistic
- 12:47Align Box Criterion
- 13:08Three Clustering Solution
related talks
47mAssessing the quality of a clustering
34mDimensionally reducing data squeezing out the good stuff
25mConstrained Rank Aggregation and Correlation Clustering
46mBayesian Optimization Accelerator
43mSpectral Approaches to Nearest Neighbor Search
24mStochastic Block Models with R: Statistically rigorous clustering
01h 02mTheory and Big Data
35mClustering: A Guide for the Perplexed