Scaling Enterprise-Grade RAG: Lessons from Legal Frontier
summary
Harvey builds a RAG stack that combines sparse and dense retrieval, domain-specific reranking, and strict privacy controls, running on LanceDB’s AI-native lakehouse and Lance format. The system indexes 3–4 billion vectors in under two to three hours, serves tens of millions of documents with low latency via compute/storage separation and GPU indexing, and enforces per-customer retention and segregation. Evaluation focuses on eval-driven development with expert labels plus automated precision/recall checks; the pipeline is production-proven across hundreds of law firms in 45 countries.
AI-written summary · about
chapters (12)
- 0:00Intro
- 0:56Overview
- 1:22What is Harvey
- 2:21Challenges
- 3:22Challenge Demonstration
- 4:38Evaluation
- 6:10Data
- 7:14Infrastructure
- 8:54LANCDB
- 11:58Single Source of Truth
- 12:38OpenLance Format
- 14:56Take home messages
related talks
15mMastering RAG and Vector Search with Elastic
31mAdvanced Enterprise RAG Systems
30mGraph RAG: Bringing together graph and vector search to empower retrieval
37mBuilding advanced RAGs with Elasticsearch OpenSearch
33mVector Databases in the Age of AI: Embracing RAG for Enhanced Search Capabilities
21mHow RAG is Redefining AI Capabilities
29mRAG for Data Engineers: Why Your Metadata Is the Missing Piece
16mOpenRAG: An open-source stack for RAG