Using Monoids for Large Scale Business Stats
summary
Claims a working system called Abel aggregates a million events in ~15 minutes on a single box, built from concrete pieces: Kafka for ingestion, local persistence in RocksDB, and Algebird-style algebraic aggregates. Metrics are implemented as commutative monoids (counts, sums, averages as sum+count pairs, HyperLogLog for uniques, count‑min sketch and q‑digest for distributions), rollups explode via Cartesian product, and queries use a scatter‑gather of partial aggregates that recombines correctly and cheaply.
AI-written summary · about
chapters (5)
related talks
38mAdd ALL the things: abstract algebra meets analytics
33mAggregating metrics in-flight: challenges and opportunities
25mMergeable Summaries for High Scale, High Cardinality Analytics
36mAggregator: MapReduce in the type system
30mLightweight Business Intelligence
17mOptimizing Data-Driven Decisions
48mHyper Speed! When Big Data Blooms
45mReal Time Analytics with MongoDB