Machine Learning with Clojure and Apache Spark
summary
Compares Clojure bindings for Spark by building a binary classifier on 600,000 LA police stops using flambo vs sparkling, trains a decision tree (max depth 5, 32 leaves) and measures ~77% accuracy on test data versus ~81% with boosted trees; then builds a convolutional network with DeepLearning4J for animal images (4 classes), gets modest precision/recall/F1 (~0.375/0.375/0.3529), notes heavy JVM interop and the need to prune and guard against biased training data.
AI-written summary · about
chapters (19)
- 0:00Intro
- 1:20About Me
- 2:43Agenda
- 4:30What's Supervised Learning?
- 5:31What's Apache Spark?
- 6:40Some Spark Terminology
- 7:35Our Data
- 8:35Features && Labels
- 9:57Decision Trees
- 13:28Part 2: A Tale of Two DSLS
- 14:03Flambo Example
- 14:49Sparkling Example
- 15:39Straight Spark
- 20:48What's Deep Learning?
- 26:55Max Pooling/Downsampling
- 28:10Alternating Layers
- 32:33How'd We Do?
- 34:19Summary
- 35:55Takeaways (TL;DPA)
related talks
50mApache Spark vs Machine Learning
26mDistributed Computing in Clojure and Spark
28mMachine Learning Live
41mUsing Clojure+Spark to Find All the Topics on the Interwebs
39mMachine Learning with Scala on Spark
29mDeep Learning Needs Clojure
53mA Taste of Random Decision Forests on Apache Spark
54mDeep Learning with Apache Spark