Read as Needed: Building WiSER, a Flash-Optimized Search Engine
summary
WiSER is a clean-slate, 11k-line C++ search engine that read as needed from high-performance SSDs using four concrete techniques: grouping by term layout, a two-way cost-aware Bloom filter, adaptive prefetching, and per-document compression. Evaluation on a Wikipedia-based benchmark (≈6M docs) with 0.5 GB RAM shows WiSER trades up to 50% more storage to cut read amplification by up to 3×, raise query throughput up to 2.7×, and cut latency by 16× versus Elasticsearch.
AI-written summary · about