BloomIntent: Automating Search Evaluation with LLM-Generated Fine-Grained User Intents
summary
BloomIntent automatically expands ambiguous queries into LLM-generated fine-grained intents grounded on taxonomies derived from Naver search logs, runs an automated LLM-based evaluation per intent, clusters similar intents, and surfaces results in a structured interface. Three technical evaluations show generated intents are clearer and more realistic than baselines, automated intent-level satisfaction agrees with expert judges at 72% agreement, and a small case study (N=4) finds the tool uncovers underserved intents but currently handles single-turn static intents.
AI-written summary · about
chapters (7)
- 0:00Introduction and motivation
- 1:30Problem with query ambiguity
- 3:21Intent-based evaluation approach
- 5:09BloomIntent method overview
- 6:14Scalable evaluation pipeline
- 7:39Case study with search experts
- 9:04Future directions for multi-turn search
related talks
43mAI enhanced UX
22mAgentic Hybrid Retrieval: Orchestrating LLM Agents, Hybrid Search & Explainability in OpenSearch
32mBeyond Vibe Coding: Leveraging Foundation Models to Infer Intent
29mGenerative-AI: Usecase-Specific Evaluation of LLM-powered Applications
46mLLM—Search Engine Killer?
50mBeyond the Prompt: Evaluating, Testing, and Securing LLM Applications
15mSearch Engineering Reimagined: Leveraging LLM in ML Workflows
09mFine-tuning embeddings 4 vector search