Elasticsearch-Quality Full-Text Search in Postgres via Tantivy
summary
Demonstrates a Postgres extension, pg_search, that embeds the Tantivy search engine (built in Rust with pgrx) to add BM25 scoring, richer tokenization, fast faceting and analytics, and a SQL-accessible score projection. Implements a new @@@ operator, a custom BM25 index (one index per table), predicate pushdown, and columnar “fast” fields so COUNT and facets run orders of magnitude faster; uses paradedb.score() to return BM25 values and claims Tantivy twice as fast as Lucene for the core search work.
AI-written summary · about
chapters (10)
- 0:00Introduction
- 2:50About ParadeDB
- 4:58About is Full Text Search (FTS)
- 11:25What Postgres Full Text Search is missing
- 13:54What is BM25?
- 16:02Introducing pg_search
- 16:42About Tantivy
- 18:25How pg_search improves full text search
- 25:41How pg_search improves aggregates/facets
- 27:24Use cases and conclusion
related talks
27mMulti-table Full Text Search with Postgres
34mFull-Text Search in PostgreSQL
43mFull-Text-Search Explained from A to Z with French Food
24mSearch From the Ground Up
42mYou don't need Elasticsearch! Fuzzy Search with PostgreSQL and Spring Data
21mTorus: PostgreSQL Search in Ecto
59mFull-text search - seek and ye shall find
41mPostgreSQL is (probably) all you need