Layering every technique in RAG, one query at a time
summary
Starts with in-memory embeddings and moves stepwise through BM25, relevance embeddings, cross-encoder re-rankers, custom embeddings and domain ranking signals, then adds user-preference signals, query orchestration (fan-out), supplementary retrieval, distillation and graceful degradation. Gives concrete limits — chunking can be disastrous, keyword queries sometimes beat vectors, ambiguous queries like “falafel” need multi-backend recall — and claims a final planet-scale system: 70+ corpus mix of tokens, embeddings and knowledge graphs, jointly retrieved and re-ranked to reach 160,000 QPS under 200msec.
AI-written summary · about
chapters (13)
- 0:00Introduction and Context
- 1:41Quality Engineering Loop and Mindset
- 4:09In-Memory Retrieval
- 4:50Term-Based Retrieval (BM25)
- 5:18Relevance Embeddings (Vector Search)
- 6:15Re-Rankers (Cross Encoders)
- 7:59Custom Embeddings
- 9:40Domain-Specific Ranking Signals
- 11:09User Preference Signals
- 12:17Query Orchestration (Fan Out)
- 14:26Supplementary Retrieval
- 16:09Distillation
- 17:14Punting the Problem and Graceful Degradation
related talks
02h 47mFrom naive to advanced RAG: the complete guide
37mBuilding advanced RAGs with Elasticsearch OpenSearch
26mBeyond the Vector DB
28mEmbeddings
42mResearch in the Age of AI
35mThings I Learned the Hard Way Building a Search Engine
20mDataEngBytes 2023 - PER-T1-06 - Kamal Hossain
39mCross Domain Enterprise Search - Content Diversity at Scale