Optimizing AI: From RAG to Riches!
summary
Gives concrete, actionable items: a four-stage RAG system (filter by metadata, vector search, rescore, rerank) with a movie DB example filtering IMDb > 7; concrete token savings via prompt compression (examples compressing ~50,000 tokens down to ~10,000 while preserving output); storage and latency tradeoffs with vector quantization and numbers (10 million 768‑dim vectors ≈ 30 GB RAM, 10 million 1536‑dim ≈ 60 GB), plus when to use hybrid lexical + vector search.
AI-written summary · about
chapters (10)
- 0:00Talk overview
- 1:13LLMs and RAG limits
- 2:39Vectors and embeddings
- 5:28Building RAG pipeline
- 7:16Enhancing RAG with filters
- 9:02Four-stage RAG system
- 11:58Cost savings with prompt compression
- 15:14Reducing vector size via quantization
- 19:25Key takeaways
- 20:08Q&A
related talks
19mRAG in 2025: State of the Art and the Road Forward
27mOptimizing RAG Systems in Practice: Challenges and Proven Solutions
13mRAG Without the Hassle: Building AI-Powered Applications
06mWhat makes vector databases essential for RAG and AI apps?
01h 35mTutorial: Build AI-Supercharged RAG Apps with a Vector Database
33mVector Databases in the Age of AI: Embracing RAG for Enhanced Search Capabilities
44mRAG vs Fine tuning which one is better for your AI project
25mAdvanced RAG Systems: Scaling Retrieval-Augmented Generation in Production