When Probably is Good Enough
summary
Examines Redis Stack’s bloom filter, count min sketch, and top K with concrete numbers and a demo: Pride and Prejudice text and an 813,695-usernames file. Bloom uses 1.17 MB vs a 46 MB set and drops ~400 names (~0.05%), Top K reports 4073 vs true 4078 and count‑min sketch overestimates but stays close. Covers cuckoo filters, HyperLogLog, T‑digest, and real uses (Chrome cache, Adobe de‑duping, Akamai cache gating) and the tradeoffs of space, speed, and accuracy.
AI-written summary · about
chapters (10)
- 0:00Introduction to Redis
- 1:29Why use probabilistic data
- 3:43Defining probabilistic data
- 6:14Bloom filters explained
- 9:24Count-min sketch mechanics
- 13:36Top-K data structure
- 17:56Additional structures
- 19:52Real-world use cases
- 23:12Live demo and comparison
- 31:58Q&A and closing thoughts
related talks
01h 00mCount-min Sketch to Infinity
48mProbabilistic data structures
51mProbabilistic Data Structures with Redis Bloom
01h 00mHyper Speed: When Big Data Blooms
10mDon’t forget to sketch! Running with large datasets
26mNo, Maybe and Close Enough! Probabilistic Data Structures with Python and Redis
39mAcceptably Inaccurate: Probabilistic Data Structures #bbuzz
25mHigh on Cardinality, Low on Memory