LLMs gone wild
summary
Shows how retrieval augmented generation is built end-to-end: chunking Markdown docs, creating embeddings with OpenAI, storing them in Asher AI Search, and wiring a LangChain RAG that surfaces chunks in a Chainlit UI — the demo answers QA but still hallucinates. Gives concrete optimizations: markdown/recursive splitters, vector database indexing, metadata filtering, query expansion, hybrid search, reranking, hypothetical embeddings and agents (search + calculator). Cites rough accuracies (retrieval and generation ~60–85% each, compounded ≈72%), urges evaluation metrics (context precision/recall, faithfulness, relevancy) and guardrails for production.
AI-written summary · about
chapters (18)
- 0:00Introduction
- 1:27LLM limits with facts
- 5:45What is RAG
- 7:54Embeddings in RAG
- 16:22RAG demo with LangChain
- 22:47Demo to production challenges
- 28:28Evaluating RAG systems
- 33:46Optimizing RAG ingestion
- 39:00Advanced document chunking
- 41:28New embedding and retrieval
- 44:25Metadata filtering
- 47:16Query expansion and hybrid search
- 49:45Reranking and prompt tuning
- 50:51Agents for RAG
- 53:43RAG benchmarking and accuracy
- 57:18When RAG fails and safety
- 58:44Guardrails for RAG
- 59:48Conclusion
related talks
57mLLMs gone wild
50mRetrieval Augmented Generation Beyond Toy Demos
48mGoing Beyond Prompt Engineering
27mUnlocking the Potential of Large Language Models
01h 33mRAG How To Talk To Your Data
01h 14mAn Introduction to Retrieval Augmented Generation
28mNavigating Challenges and Enhancing Performance of LLM based Applications
53mFrom RAG to Riches: Secrets of Data-Augmented LLMs