Implementing the k-means jump method
summary
Implements the jump method on the Wisconsin breast‑cancer set (~650 samples, 9 attributes) using Clojure, Incanter, clj-ml (Weka), calculating raw distortion (sum of squared errors) for k=1..n, applying transform distortion = distortion^(-2), then computing pairwise jumps (difference between successive transformed distortions). The largest jump occurs at k=2, producing two clusters that nearly match benign vs malignant; includes a runnable REPL implementation, bootstrapping discussion, and a clj-ml patch for k-means initialization.
AI-written summary · about
chapters (7)
- 0:00Start
- 7:50The Elbow Method
- 9:54Finding Number of Clusters in the Data Set an Information Theoretic Approach
- 10:47Rate Distortion Theory
- 12:44The Jump Method
- 13:51Calculate the Transform Distortion
- 23:35Tooling
related talks
35mClustering: A Guide for the Perplexed
01h 00mStreaming Euclidean k-median and k-means with o(log n) Space
19mtidyclust - expanding tidymodels to clustering
27mKubernetes + AI Joining Forces in the Battle Against Cancer
17mKubism: Disassembling and Reassembling K-Means Clustering for Mobile Heterogeneous(…)
47mAssessing the quality of a clustering
42mk-means and k-medians under Dimension Reduction
40mTools and Tricks from a Pragmatic Data Scientist