Building semantic search using Deep Learning
summary
Semantic search replaces token matching with vector similarity: a REST ML service holds precomputed vectors for ~1.3 million companies (trained on ~4 million news articles), computes a query paragraph vectors encoding, does cosine similarity and returns IDs above a tuned threshold; an ElasticSearch proxy injects those IDs back into the existing ES faceted stack. Results jump (example 76 vs 22), model runs on CPU using 8–12 GB memory, 300‑dim vectors initialized from GloVe, retrained monthly.
AI-written summary · about
chapters (14)
- 0:00Introduction to semantic search
- 1:57What Mattermark does and NLP needs
- 4:06Traditional search limitations
- 6:56Why build a custom semantic search
- 8:41What is semantic search
- 9:25Live demo of new system
- 14:08Choosing the right approach
- 16:39Vector space models tested
- 19:26Paragraph vectors solution
- 20:52Integrating ML with Elasticsearch
- 25:13System architecture overview
- 26:05Training data and query vectors
- 28:35Hyperparameter tuning details
- 31:48Closing remarks and hiring
related talks
22mThe Capabilities of Semantic Search
31mAdvanced search for your e-commerce website
13mA Beginner’s Guide to Semantic Search
39mAI-powered Semantic Search; A story of broken promises?
30mSearch on Steroids — A Story of Semantics
30mSemantic Search and Neural Information Retrieval
33mA Practical Approach To Semantic Search
01h 28mBuilding a Semantic Search Engine