Measuring the Performance of VectorRAG vs GraphRAG
summary
Comparison uses RAGAS automated assessment on Neo4j graphs built from movie (≈100 Q/A) and product (≈250 Q/A) datasets, with questions expanded by LLMs and human-validated. Three retrievers are measured: plain vector search, augmented Vector search (vector + Cypher traversal), and text-to-Cypher. Results show graph RAG methods beat vector-only on retrieval and generation metrics; text-to-Cypher is the most consistent; augmented vector sometimes hallucinates numeric aggregates but can return partial correct answers; vector search works well only for node-property queries.
AI-written summary · about
chapters (11)
- 0:00Introduction and agenda
- 1:22Understanding RAG
- 2:46Evaluation motivations
- 4:27Vector and Graph RAG strategies
- 7:31Retrieval strategies details
- 9:07The Ragas framework
- 13:16Experimental setup and data
- 18:47Evaluation results
- 19:39Case study examples
- 21:09Performance summary
- 21:57Conclusion and future work
related talks
40mRAG Pipeline Evaluation
27mWhen Vectors Aren’t Enough: Unlocking AI’s Hidden Connections with Graphs
16mEnhancing Retrieval-Augmented Generation with GraphRAG Patterns in Neo4j
39mEnhancing RAG with Neo4j Knowledge Graph
20mHybridRAG: A Fusion of Graph and Vector Retrieval
30mAgentic RAG with Knowledge Graphs Explained
36mAgentic GraphRAG: AI’s Logical Edge
31mEnhancing RAG with Fast GraphRAG and InstructLab: A Scalable, Interpretable, and Efficient Framework