Make Your Data FABulous
summary
Elasticsearch tradeoffs center on the FAB: Fast Accurate Big tradeoff: distributed shards give fast, big queries but can hurt accuracy. Example: a terms aggregation returns 56 counts instead of 64 because each shard returns top N where N = shard_size = size1.5+10, dropping lower-frequency terms. Score differences come from per-shard IDF; setting DFS=true forces a cluster-wide term frequency at the cost of an extra round trip. Workarounds include shrinking to one shard or using the new single-shard default in 7* and split/shrink APIs to rebalance.
AI-written summary · about
chapters (9)
- 0:00Start
- 1:54What is the perfect datastore solution?
- 3:10Questions: Answers
- 10:15Big Parallel computing tools are needed to handle the data
- 10:38The 42 V's of Big Data and Data Science
- 13:57Terms Aggregation
- 18:28Routing shard# = hash(_routing) % #primary shards
- 26:36Term Frequency / Inverse Document Frequency (TF/IDF)
- 32:30DFS Query Then Fetch Distributed Frequency Search
related talks
48mMake Your Data FABulous
33mMake Your Data FABulous
41mTips and Tricks to Scale Elasticsearch for 1M RPMs and Beyond
37mBeyond the basics with Elasticsearch
37mAlgorithms & data-structures that power Lucene & ElasticSearch
48mAnalytics Architectures on Microsoft Fabric
40mWhat's evolving in Elasticsearch and Lucene?
29mHow (not) to Scale Elasticsearch for Data Analytics!