Data Science in Clojure

on YouTube ↗44m

" summary

Yieldbot builds its data platform in Clojure and processes about 1 billion page views a week using Kafka, Storm/Trident and Spark. The team replaces Hadoop with a streaming + Spark stack and ships two open source DSLs: marceline (a Clojure Trident layer with state management backed by Cassandra) and flambo (a Clojure DSL over Spark that supports REPL-driven jobs via serialized functions). Both run in production, speed prototyping-to-deployment, and handle large-scale text processing and online learning.

AI-written summary · about

" chapters (12)

1. 0:00Intro
2. 0:32About YieldBots
3. 2:18Dealing with Data Science
4. 2:55Machine Learning
5. 4:54JVM
6. 7:48Hadoop
7. 10:36Storm
8. 12:06Storm Topology
9. 15:17Spark
10. 24:56Why Clojure
11. 26:36Closure Demo
12. 30:43Opportunities

" related talks

• 43mDoing Data Science with ClojureSimon Belak · Lambda World 2016
• 18mClojure Versus the BotnetsMarshall Bockrath Vandegrift · Clojure/conj 2015
• 26mDoing data science with Clojure: the ugly, the sad, the joyfulSimon Belak · clojureD 2016
• 40mClojure Emerging Data EcosystemSami Kallinen, Ethan L. Miller · re:Clojure 2021
• 40mLessons Learned Creating a Successful Machine Learning Startup with ClojureAlex Hudek · Clojure/north 2019
• 38mScientific Clojure, a bird's eye viewThomas Clark · Clojure/conj 2024
• 41mA Clojure fusion of symbolic and data driven AIHuahai Yang · Clojure/conj 2018
• 27mData and Clojure: Using Clojure for more than software developmentKira McLean · reClojure 2022