Understanding Probabilistic Data Structures with 112,092 UFO Sightings
summary
Explains probabilistic data structures by loading 112,092 UFO sightings into Redis via the RedisBloom module, then demonstrates membership testing with a bloom filter (fixed-size bit array, multiple hash seeds, false positives when full) and heavy-count ranking with top-k (heavy keeper + min-heap) (hash-based counters with probabilistic decay). Shows Python code that parses the CSV, issues bf.reserve / bf.add and topk.reserve / topk.add commands, and runs a live updater that successfully surfaces the top UFO words and shapes despite brief demo glitches.
AI-written summary · about
chapters (11)
- 0:00Introduction and theme
- 1:18Talk overview and data source
- 7:18What are probabilistic data structures?
- 9:44Hash tables and deterministic structures
- 16:46Accuracy trade-offs in data structures
- 20:05Types of probabilistic data structures
- 24:56Bloom filters explained
- 34:01Bloom filters in Redis
- 36:50Top K and heavy keeper
- 44:15Demo and resources
- 46:42Q&A and closing
related talks
53mUnderstanding Probabilistic Data Structures with 112,092 UFO Sightings
51mProbabilistic Data Structures with Redis Bloom
37mWhen Probably is Good Enough
39mAcceptably Inaccurate: Probabilistic Data Structures #bbuzz
48mProbabilistic data structures
41mProbabilistic Data Structures
01h 50mData sketching workshop
22mData Exploration Made Easy with Open Source