Comparing vector implementations in generic databases
summary
Compares pgvector on Postgres versus Lucene-based vectors in OpenSearch/Elasticsearch using a real RAG case: 5 million vectors, ~1,645 dimensions, ~1,000 tenant namespaces and a 200 ms latency budget. Shows OpenSearch KNN plugins force huge in-memory HNSW graphs causing circuit breakers and high cost, while Lucene now supports up to 16k dims and on-the-fly prefiltering. Shows Postgres using partitioning or per-tenant branches plus binary quantization or 2-byte floats and top-k rerank (e.g. k=800) wins on cost while meeting recall.
AI-written summary · about
chapters (11)
- 0:00Introduction to talk
- 1:05Why vector support matters
- 3:34Vector support across databases
- 5:01Pros and cons of generic vs specialized
- 7:13Case study with customer RAG use case
- 9:20Testing OpenSearch for vector search
- 12:12Postgres and PG Vector approach
- 13:35Zeta branches for scalability
- 14:37Improvements in six months
- 17:50PG Vector quantization methods
- 20:39Conclusion and summary
related talks
41mPostgreSQL for AI. Real Vector Search in PostgreSQL
52mVector search is now boring, but PostgreSQL has ways to go
48mFrom Text to Meaning: pgvector Transforms PostgreSQL Search
44mScaling Semantic Search in PostgreSQL
52mpgvector Newbie to Expert
33mBeyond Benchmarks: Real-World AI Search Performance in OpenSearch (Lucene Vs Faiss Vs JVector)
35mImpact of Vector Search
04mSupercharging GenAI PostgreSQL Vector Search Optimization