Teaching Ruby to Rank: A Search Pipeline Story
summary
Fullscript builds a full Learning to Rank pipeline entirely in Ruby that takes click and impression logs, auto‑engineers BM25 and relative rank features, trains XGBoost models, and reranks OpenSearch hybrid retrieval candidates to improve ecommerce relevance. The system runs daily training and optimization, serves up to four engines (autocomplete, regional splits), shows clear NDCG gains, and drives negative search feedback nearly to zero. They use pure Ruby pipeline tooling and plan to release the code as a gem.
AI-written summary · about
chapters (16)
- 0:00Teaching Ruby to rank
- 1:26Search pipeline at Full Script
- 3:38BM25 scoring in OpenSearch
- 5:22Static query boosting limits
- 6:03User feedback on search failures
- 7:07Measuring relevance with NDCG
- 10:49Learning to rank in Ruby
- 11:33Training data from user behavior
- 12:55Feature engineering
- 14:41Types of ranking features
- 15:26Optimizing model features
- 16:50Deploying reranking model
- 18:56Results and feedback
- 19:37Lessons learned
- 21:04Future plans for Ruby gem
- 22:07Q&A
related talks
31mSponsored Session: From Clicks to Rankings: Building a Learning-to-Rank Pipeline with…
47mLearning to rank search results
43mLearning to Rank for Reddit Search: A Project Retro
38mLearning to Rank: where search meets machine learning #bbuzz
55mLearning to Rank, a Little
43m[Machine Learning] Creating more relevant search results with Learn to Rank
40mLevel Up: Everything you need to know about search, ...
38mLearning to Rank journey at GetYourGuide: Our Logbook #bbuzz