Optimizing RAG with Semantic Caching & LLM Memory
summary
Claims that semantic caching plus LLM memory cut cost and latency by reusing RAG outputs, with empirical numbers: ~30% question redundancy, ~60% cache hit rate, and >90% positive hits on cache matches. Demonstrates fine-tuning embeddings to raise semantic-match quality and speed, publishes the model on Hugging Face, and ships an open source agent memory server that summarizes chat into dense long‑term facts. Also releases a LangCache developer API and a Redis‑backed vector library and cookbook.
AI-written summary · about
chapters (9)
- 0:00Introduction
- 1:24Optimizing RAG systems
- 3:31Semantic caching for LLM queries
- 5:57Evaluating cache hit rates
- 7:00Fine-tuning embeddings
- 8:24LLM memory limitations
- 10:11Short-term vs long-term memory
- 11:34Future of memory frameworks
- 12:37Open-source memory server
related talks
30mDesigning Fast & Cost-Efficient Agents: The Power of Caching
20mReal-time RAG: How to augment LLMs with Redis and Amazon Bedrock
14mSmarter RAG, Smaller Bill: Optimize for Performance and Price
59mThe Evolution of RAG
27mOptimizing RAG Systems in Practice: Challenges and Proven Solutions
48mCrafting your own RAG system: Leveraging 30+ LLMs for enhanced performance
28mNavigating Challenges and Enhancing Performance of LLM based Applications
19mRAG in 2025: State of the Art and the Road Forward