RAG: Tricks from the trenches.
summary
Practical RAG lessons: changing one embedding model raises relevant-retrieval from about 16.5% to 62.89% (roughly a 300% gain), so embedding choice matters more than which vector DB is used. Examples use SentenceTransformers, LanceDB for a 25-line RAG, scikit-learn clustering for representative summaries, GPT-3.5 Turbo with Pydantic-style structured outputs for verifiable facts, and BM25 proved superior in one eight-month production rollout. Production advice: start simple, log queries and contexts, iteratively fix the hardest failures.
AI-written summary · about
chapters (9)
- 0:00Introduction and speaker background
- 1:49What is RAG and why it matters
- 6:23Why your embedding choice sucks
- 9:38Structured outputs for sanity
- 14:33Exploding context and summaries
- 18:47Ugly production data realities
- 21:16Is your system actually any good?
- 24:09Final thoughts and resources
- 25:13Q&A session
related talks
19mRAG in 2025: State of the Art and the Road Forward
28mNavigating Challenges and Enhancing Performance of LLM based Applications
56mRAG: from dumb implementation to serious results
44mBuild RAG from Scratch
53mBuild RAG from Scratch
27mOptimizing RAG Systems in Practice: Challenges and Proven Solutions
49mRAG: from dumb implementation to serious results
25mAdvanced RAG Systems: Scaling Retrieval-Augmented Generation in Production