Achieving Low-Latency Graph-Based Vector Search via Aligning Best-First Search...
summary
An on-disk system called PipeANN aligns best-first graph search with SSD behavior by issuing speculative, pipelined reads decided from the in-memory candidate pool, dynamically widening the IO pipeline during a converge phase and delaying some reads to cap wasted neighbors. Evaluation on 100M and billion-scale datasets shows pipe search achieves 1.14×–2.02× the latency of in-memory Vamana and only 35.0% of DiskANN latency at comparable recall, while keeping throughput near other on-disk indexes.
AI-written summary · about
chapters (12)
- 0:00Low-latency vector search motivation
- 1:21Vector search basics and ANN methods
- 2:49SSD latency challenges in vector search
- 3:32Latency mismatch between SSD and graph search
- 4:57Data dependency in best-first search
- 5:59Pipe Search to break compute-IO dependency
- 7:26Pipe Search compute-IO overlap
- 8:31Pipe Search two-phase approach
- 9:55Handling simultaneous IO completions
- 10:40PipeIn system overview and benefits
- 11:03PipeIn vs existing methods results
- 11:45Summary and open-source
related talks
16mTowards High-throughput and Low-latency Billion-scale Vector Search via CPU/GPU...
18mScalable Billion-point Approximate Nearest Neighbor Search Using SmartSSDs
01h 09mVector Search: Ask Me Anything!
40mUnder the hood of vector search with JVector
44mVector Search Beyond the Hype: Real-World Implementation and Performance Insights
14mFast, Approximate Vector Queries on Very Large Unstructured Datasets
19mHow Microsoft powers planet-scale AI apps with DiskANN
32mRetrieval Augmentation and Semantic Search at Scale