Building High Performance Search Indexes in Rust with SIMD
summary
Claims state-of-the-art sparse and dense retrieval using Rust and careful SIMD work, combining IVF + product quantization to avoid exhaustive scans and hitting 40–50 ms on hundreds of millions of vectors. AVX2 auto-vectorization plus alignment and u8→u64 XOR/popcount give a 6x speedup; hand-tuned AVX‑512 intrinsics and register shuffles yield about a 3x improvement over naive PQ. Tooling includes Cargo Flamegraph, Rayon for parallelism, zero-copy blobs, per-CPU builds (Ice Lake vs Cascade Lake) and a note that AVX‑512 is still unstable in Rust while AMX support is desired.
AI-written summary · about
chapters (36)
- 0:00Introduction
- 0:20About Jack
- 1:26What is a vector embedding
- 2:06What is a vector database
- 2:48Whats required for a vector database
- 3:58Existing Search Engines
- 4:31Choosing a Language
- 4:52Why Rust
- 5:33What my team does
- 5:48Dense vs sparse indexes
- 7:25How to optimize indexes
- 8:08Early termination
- 9:10Ignoring posting list
- 9:35Results
- 9:52CargoFlame Graph
- 10:44Release Mode
- 11:12SIMD is not a silver bullet
- 11:28Dense ANN algorithms
- 11:42Compression
- 13:16Query Execution
- 14:56Tools
- 16:33Interactive Code
- 17:09AVX2 Alignment
- 17:25Convert Slices to U64
- 17:48IVF
- 19:19Product Quantization
- 22:12Intrinsics
- 23:10Using Intrinsics
- 25:28How much does this help
- 26:04Is it maintainable
- 26:43Zerocopy data format
- 27:17Engineering team
- 27:53Rust struggles
- 28:17Rust is safe
- 28:41Memory utilization
- 29:13Questions
related talks
32mRetrieval Augmentation and Semantic Search at Scale
25mVector Streaming: The Memory Efficient Indexing for Vector Databases
16mTowards High-throughput and Low-latency Billion-scale Vector Search via CPU/GPU...
32mUnder the Hood: Key Algorithms Powering Vector Search
59mParallel Programming in Rust: Techniques for Blazing Speed
40mRust’s approach of getting things right
19mProduction Unsafe Rust
43mOpenSearch and the Path to a Trillion-scale Vector Database