Woosh: a fast pure-Python search engine library
summary
Whoosh is a pure Python search library used to index ~12,000 product records; initial indexing runs at 10–25s for 2,000 documents producing an index ≈10 MB. The stack uses Flask then moves to Tornado after a Black Friday crash, scoring defaults to BM25F but accepts TF‑IDF and custom parsers, supports field boosting, incremental updates, filtering and paging, and stores events in MongoDB for related‑search signals. Text work requires heavy cleaning: a 5,000‑word adjective blacklist, Spanish stemming limits, and pragmatic schema choices.
AI-written summary · about
chapters (11)
- 0:00Introduction
- 2:46Why choose Woosh
- 5:16Data preprocessing challenges
- 9:17Schema and field types
- 12:52Indexing documents
- 13:58Search process and options
- 17:36Production deployment issues
- 19:04Lessons learned
- 21:14Q&A: Stemming and sorting
- 23:27Q&A: Team adoption
- 25:39Q&A: Elasticsearch vs Woosh
related talks
28mTune-Up Your Knowledge: A Beginner's Journey Under the Hood of Search Engines
30mMaster 2.5 GB of unstructured specification documents with ease
24mSearch From the Ground Up
19mSearch Solutions with OpenSearch, Python, and Serverless 🐍
36mA Python Takeover
29mFrom Java to Python: Migrating Search Functionality at billiger.de
31mA Recent History of Python Through Stack Overflow Questions and Answers
12mUsing Python to Build Applications for Language Learning