From boolean towards semantic ...
summary
A production pipeline identifies weighted query tokens as Must-have Tokens via dependency parses, trains local embeddings from catalog plus clickstream on Apache Spark, and prunes senses with centroid cosine similarity for word sense disambiguation; synonyms come from both local word2vec on catalog and clickstream and open resources, then a custom Solr query parser expands queries. Offline caching avoids runtime parsing cost. Measured gains include roughly 20% uplift in A/B tests and end-to-end synonym/must-have accuracy near 80%.
AI-written summary · about
chapters (10)
- 0:00Introduction and overview
- 1:12Problem with boolean retrieval
- 3:40Must-have tokens explained
- 5:28Query understanding process
- 9:04Synonym generation pipeline
- 11:09Word vector embeddings
- 13:58Word sense disambiguation
- 15:00Implementation summary
- 15:43Future work and conclusions
- 16:53Q&A session
related talks
35mYou've got questions. We've got answers! #bbuzz
20mLayering every technique in RAG, one query at a time
40mSmarter Search with Spark, Solr and Machine Learning
21mSmart Autocompl ...
01h 01mWord Embeddings - An Alternative & Efficient Approach to Search for Docs
26mReview Analysis: an Approach to Leveraging User-Generated...
35mThings I Learned the Hard Way Building a Search Engine
18mFree Texts to Knowledge Graphs - Holy Grail of Semantic Computing on an Industrial Scale