zoekeend: A seach engine for information retrieval experiments based on
summary
Toolchain: DuckDB with the drb full text extension, small Python experiment scripts and CF import. Claim: storing indexes in DuckDB lets experiments run with millions of documents (Robust04 ~1M) and prototype search systems that run within one coffee break. Concrete experiment: drop the stored document lengths via alter table drop column length and approximate them instead — end-to-end evaluation shows no significant loss in retrieval effectiveness, suggesting much leaner, faster IR systems are feasible.
AI-written summary · about
related talks
28mAnalyze data easily with duckdb - and the implications on data architectures
45mAnalytics for not so big data with DuckDB
25mA quack of all trades: DuckDB, a versatile analytical database to keep in your toolkit
24mData infrastructure to build bigger with less
58mData Wrangling [for Python or R] Like a Boss With DuckDB
07mAerodynamic data models: Flying fast at scale with DuckDB
47mPlay Data Engineer Locally with DuckDB
37mDuckDB: Crunching Data Anywhere, From Laptops to Servers