Challenges in Building RAG Systems (and Solutions Using OpenSearch)
summary
OpenSearch acts as the vector store using the OpenSearch kNN vector store for retrieval, while quality (not cost or latency) proves the biggest hurdle. Concrete fixes: hypothetical document embeddings (HIDE) convert a vague query into an LLM-generated answer before searching, improving precision ~10% in a 2024 study; contextual chunk headers (CCH) raised a chunk similarity from 0.1 to 0.9 and showed ~26% precision uplift in benchmarks. Also use metadata filtering (e.g., fiscal year) and try rankRAG / Raptor / GraphRAG with heavy tuning.
AI-written summary · about
chapters (11)
- 0:00RAG systems challenges
- 3:48Quality hurdle in RAG
- 4:52Data curation challenges
- 7:04Precision vs recall
- 8:06Semantic gap in matching
- 10:09Irrelevant retrieval causes
- 11:13Query transformation for gaps
- 12:37Hypothetical Document Embeddings
- 15:06Filtering irrelevant documents
- 16:52Contextual chunk headers
- 19:23Advanced RAG techniques
related talks
37mBuilding advanced RAGs with Elasticsearch OpenSearch
34mImplementing an open-source RAG with OpenSearch
45mMaking semantic search & RAG real: How to build a prod-ready app
17mFine-Grained Authorization for Secure RAG With OpenSearch
28mUse OpenSearch as Your Vector Search Database To Optimize GenAI Accuracy!
21mSmart Search with RAG: Elasticsearch Meets Language Models
17mVectors Vs. Hallucinations: OpenSearch's GenAI Survival Kit
27mOptimizing RAG Systems in Practice: Challenges and Proven Solutions