Cascalog: Making Data Processing Fun Again
summary
Cascalog is a Clojure library that emits MapReduce jobs from REPL code and runs on Hadoop or locally; the demo parses Twitter JSON about Tunisia, builds typed subqueries (reactions, properties, followers), computes an average followers count (1,281) by composing count + sum = average as a predicate macro, produces bucketed follower distributions and top‑location lists, does word counts, and finds reaction chains (length 4 and 7) with a chain-pair-simple generator. The system emphasizes reusable predicate macros, map‑side partial aggregation optimizations, and code hosted on GitHub.
AI-written summary · about
chapters (12)
- 0:00Introduction to Cascalog
- 1:13Cascalog Basics
- 4:01Query Syntax and Predicates
- 9:14Implicit Joins
- 10:26Running Basic Queries
- 14:48Aggregation Techniques
- 16:47Subqueries and Word Count
- 20:03Analyzing Tunisia Tweets
- 23:16Abstracting Data Parsing
- 29:49Advanced Querying and Macros
- 36:16Visualization and Analytics
- 41:11Advanced Graph Analysis
related talks
41mIntroducing Cascalog: Functional Data Processing for Hadoop
37mCascalog 2- Streaming MapReduce in Clojure
36mMoving Faster: Why Intent Media Chose Cascalog
01h 12mIntroduction to Cascading
37mClojure, a Sweetspot for Analytics
33mClojure Dispatches From a National Research Lab
33mClojure Data Science
37mClojure Remixed: Pyroclast Data Processing