RAGTrace: Understanding and Refining Retrieval-Generation Dynamics in Retrieval-Augmented Generation
summary
RAGTrace is an interactive system that lets users trace retrievals, inspect which documents the generator actually uses, and regenerate answers after swapping or reweighting sources; it visualizes question clusters with t-SNE heat map and problem types with a force-directed graph, shows evidence highlights, and records post-change performance. Built workflows A/B/C tie clustering, source editing, and validation; user study with 11 participants on Llama 3 + RAG over NQ reports PSSUQ 6.18 and moderate NASA‑TLX, with participants improving debugging and workflow efficiency.
AI-written summary · about
chapters (7)
- 0:00Introduction to RAGTrace
- 1:06Motivation and challenges in RAG
- 3:35Formative study insights
- 4:59System design and components
- 7:32Evaluation and user study results
- 8:55Discussion and future work
- 9:57Closing and Q&A
related talks
22mThe Power of RAG at Scale: A Year of Explosive Growth and Observab...
38mRetrieval-Augmented Generation (RAG) in NLP
25mAdvanced RAG Systems: Scaling Retrieval-Augmented Generation in Production
51mDeveloping a Custom RAG System
01mtalk about RAG
15mRAG Beyond Chatbots
39mRAGs to Riches: How Our Content Affects Retrieval Augmented Generation
31mEvaluating RAG Applications: From Retrieval to Response Quality