Retrieval Augmentation and Semantic Search at Scale
summary
Shows concrete systems and numbers: open‑source USearch and UForm libraries implemented in low‑level code using AVX512 and custom JIT metrics, with benchmarks showing 10x–18x speedups versus BLAS/NumPy and gaps of tens–hundreds× at large scale. Demonstrates serving 10 Billion vectors from a single machine, sustaining ~100k ops/sec at 100M entries, indexed 7B molecules and produced 28 billion embeddings now hosted on AWS, and explains when brute force, HNSW, SSD vs RAM, and 8/16/4‑bit quantization actually work.
AI-written summary · about
chapters (17)
- 0:00Retrieval augmentation intro
- 1:43Why RAG matters
- 3:29Hardware vs software inefficiencies
- 4:33Brute force vector search
- 6:40AVX-512 for cosine distance
- 8:51When to use HNSW
- 11:20Memory CPU bottlenecks
- 12:48OOP vector search pitfalls
- 14:57Cost efficiency of vector search
- 15:39Largest chemistry dataset
- 16:45Quantization for small models
- 20:59Hardware constraints for running large language models
- 22:44Inference engines comparison
- 24:11Small vs large models
- 26:40Where to get embeddings
- 28:49Multilingual multimodal embeddings
- 30:19Q&A
related talks
48mUnlocking Next-gen Semantic Search With OpenSearch 3.x
43mOpenSearch and the Path to a Trillion-scale Vector Database
36mWhen Semantic Search Breaks in your RAG systems: RAM Walls, Silent Failures, and the Architecture Decisions
26mOpenSearch and the Path to a Trillion-s...
46mNatural Language & Search, Improve your Search w/ Vector capabilities
01h 09mVector Search: Ask Me Anything!
32mUnder the Hood: Key Algorithms Powering Vector Search
47mWhy, What, When and How of Running Diff Types of Sear...