Querying ElasticSearch with Deep Learning
summary
Sequence-to-sequence LSTM with pointer-attention converts natural-language questions into Elasticsearch queries, trained with reinforcement learning to bypass non-differentiable DB feedback. Uses Elasticsearch, GloVe embeddings and a reward scheme (penalties for errors, +25 for correct) plus an exploration bonus that raises correct-retrieval rates by ~30%. Evaluated on the Movie Dialog dataset (≈96k questions, 17k movies); reinforcement learning reaches ~84.2% end-to-end while supervised intermediate-label training gains ~6% more; RL is sample-inefficient (12h vs 1h training) and handles ambiguity (~4% ambiguous entities) but works only to a degree.
AI-written summary · about
chapters (9)
- 0:00Intro
- 2:38The Problem
- 8:55Deep Learning
- 13:40Model
- 15:19Rewards
- 22:19Reinforcement Learning
- 23:51Soft Lookup vs Hard Lookup
- 24:58Summary
- 27:33Questions
related talks
39mExperiments with OpenSearch and AI
51mQuery Understanding With Large Language Models: Techniques and...
21mUsing Deep Reinforcement Learning for Dialogue Systems
01h 30mUsing LLMs to improve your Search Engine
24mConversational Search with LLMs
43mHarnessing DeepSeek-R1 and OpenSearch for RAG-Powered Intelligent Info...
21mSmart Search with RAG: Elasticsearch Meets Language Models
30mBeyond Keywords: Building an AI-Powered Search That Reads Your Mind