Understanding Probabilistic Data Structures with 112,092 UFO Sightings
summary
Dataset of 112,092 UFO sightings (scraped by Timothy Renner from the National UFO Research Center) backs concrete demonstrations: Bloom filter via RedisBloom BF Reserve for O(1) membership checks, MinHash with shingled trigrams to compute Jaccard similarity, and a TopK heavy keeper (width/depth/decay tuned) to stream-rank words. A Python build uses pandas to load CSV into Redis; a JS frontend polls Redis to show rankings. Counts are deliberately approximate but rankings remain stable; surprising findings include ~80% of sightings within 100 km of USAF bases and a Fourth of July spike.
AI-written summary · about
chapters (11)
- 0:00Introduction and welcome
- 3:18UFO sightings dataset overview
- 8:40Deterministic vs probabilistic data structures
- 15:23Accuracy trade-offs in data structures
- 20:19Probabilistic data structures in Redis
- 22:49Bloom filters explained
- 31:20Minhash and Jaccard similarity
- 35:55Top K and heavy keeper structures
- 42:57Live demo of Top K in Python
- 46:04Resources and GitHub repository
- 48:12Q&A session
related talks
55mUnderstanding Probabilistic Data Structures with 112,092 UFO Sightings
59mUnderstanding Probabilistic Data Structures with 112,092 UFO Sightings
53mUnderstanding Probabilistic Data Structures with 112,092 UFO Sightings
26mNo, Maybe and Close Enough! Probabilistic Data Structures with Python and Redis
51mProbabilistic Data Structures with Redis Bloom
26mNo, Maybe and Close Enough: Using Probabilistic Data Structures in Python
37mWhen Probably is Good Enough
01h 00mHyper Speed: When Big Data Blooms