Full-Text Search Internals with Elasticsearch
summary
Live demo uses Elasticsearch + Apache Lucene with a Kibana console and a Docker compose demo to index three example documents, show tokenization, lowercasing, stop‑word removal, stemming and synonym replacement, and then query with fuzziness (auto: 0 for 1–2 chars, 1 for 3–5, 2 for >5). Scoring uses BM25 scoring (term frequency, inverse document frequency, field length norm) and supports phrase slop, n‑grams, multi‑analyzers, reindex API and scripted function scoring.
AI-written summary · about
chapters (22)
- 0:00Intro
- 0:11Full-Text Search Internals with Elasticsearch
- 0:27elastic Developer Advocate
- 4:19Store
- 6:11Example These are not the droids you are looking for.
- 6:33standard Tokenizer
- 7:03lowercase Token Filter
- 7:25stop Token Filter droids you looking
- 8:26snowball Token Filter
- 14:12German Stop Words
- 18:14Another Example
- 29:22Term Frequency / Inverse Document Frequency (TF/IDF) Search one term
- 32:59Vector Space Model Search multiple terms
- 34:18Coordination Factor Reward multiple terms
- 34:46Practical Scoring Function Putting it all together
- 35:13Function Score Script, weight, random, field value, decay (geo or date)
- 36:43Compare Scores "100% perfect" vs a "50%"match
- 39:23Performance
- 40:51Conclusion
- 41:06Advanced Queries Highlighting NGrams & Edge Grams Multiple Analyzers Reindex & Alias
- 44:05Trainings
- 44:55Thank You! Questions?
related talks
55mFull-Text Search Demystified
25mApache Lucene: From Text Indexing to Artificial Intelligence
26mFrom __icontains to search
37mAlgorithms & data-structures that power Lucene & ElasticSearch
35mElasticsearch - Beyond Full-text Search
01h 04mApache Lucene 8, Apache Solr and Elasticsearch – an Overview
45mAnything New in Search? What If I Find It?
28mSuper Search with OpenSearch and Python