So Let’s Build a Data Indexing System
summary
Describes 2GIS’s indexing system for search data that needs very frequent updates (some segments every 10 minutes). Explains how they deliver fresh data quickly to the search engine so users see up-to-date results, focusing on the pipeline and techniques used to reduce latency between data change and search availability.
AI-written summary · about
chapters (14)
- 0:00Introduction
- 1:50Company and data
- 5:25Old system problems
- 8:12New system requirements
- 11:23Solution architecture search
- 13:32Custom solution choice
- 14:55New system architecture
- 16:40File Storage component
- 18:30Controller role
- 19:56Indexer cluster
- 20:39Syncer functions
- 22:06Monitoring and alerts
- 23:09Summary and benefits
- 24:56Q&A
related talks
24mReimagining Search : Scaling OpenSearch Across an Enterprise - Platform C...
44mFuzzy Search in the Browser Over Half a Million Lines: Building an Index on a CDN
44mBuilding a Modern Search Engine: The Road Ahead
44mHow to keep client-side data always up to date
33mExponential Growth Without Sacrificing Platform Speed
31mNext-Gen Search: How Uber and the OpenSearch Community Built a Cloud Native 3.0
39mRelevant search results (with elasticsearch)
01h 13mMobile Search