Cascalog 2- Streaming MapReduce in Clojure
summary
Cascalog 2 compiles Datalog-style queries into a pluggable logical graph and can target Hadoop, an in-memory planner, or Storm Trident, letting the same word‑count and join queries run as a Hadoop job or as low-latency streams; the rewrite adds a planner API, graph-level optimizations (for example pruning unused fields), and local in-memory mode plus Trident (Storm) mode, and it supports serializable anonymous functions — the new compiler runs in the REPL and the release is available.
AI-written summary · about
chapters (13)
- 0:00Intro
- 1:26What is Cascalog
- 4:32Cascalog code
- 5:25MapReduce wordcount
- 8:02Cascalog overview
- 9:49Subquery macro
- 12:31Example
- 14:44Abstractions
- 16:39Open Source
- 17:33Why Cascalog
- 20:18How Cascalog works
- 27:21Cascalog on other platforms
- 28:25Conclusion
related talks
41mIntroducing Cascalog: Functional Data Processing for Hadoop
46mCascalog: Making Data Processing Fun Again
36mMoving Faster: Why Intent Media Chose Cascalog
33mClojure Dispatches From a National Research Lab
31mLightning Talks
46mClojure Scaling the Event Stream
26mDistributed Computing in Clojure and Spark
37mClojure Remixed: Pyroclast Data Processing