Scaling Facets to the Stars 🌟
summary
Explains how Apache Solr range facets are engineered to return daily counts over years at sub-second latency by prewarming the filter cache and aligning query time boundaries; uses Apache JMeter for repeatable load tests and Eclipse Memory Analyzer to reverse‑engineer filter cache contents from heap dumps; reports 100% cache hit rate after warming, scales from ~10 queries/minute to 10 queries/second and cuts latency from tens of seconds to under one second on a corpus of 3 billion documents.
AI-written summary · about
chapters (15)
- 0:00Introduction and context
- 0:39The faceting challenge
- 1:57Why Apache Solr?
- 3:09Understanding faceting
- 5:42Performance challenges
- 6:49Basic system components
- 8:25Deep dive into doc values
- 10:11Experimental process
- 12:24Initial performance analysis
- 14:40Reverse engineering with heap
- 16:59Analyzing cache mechanics
- 18:48Optimization strategies
- 23:25Memory optimization steps
- 26:26Final results and takeaways
- 28:03Q&A session
related talks
33mRelevant Data Analysis: Apache Solr Analytics #bbuzz
20mGain speed and space with NLP in Solr #bbuzz
29mBuilding Analytics Applications (...)
27mTales at Scale: Analytics at 1000 QPS and Beyond
49mCounting things at the speed of light with roaring bitmaps
44mProcessing large data with Apache Spark
53mVisualizing the Output of Your Search AI
30mScaling Grandma’s Hefezopf