How to look at your data
summary
Argues for fast evals: cheap golden query–document pairs that run in seconds to measure retrieval and compare embedding models on real data. Shows a Weights & Biases case where recall@10 ranks models differently (text-embedding-3-small performs worst, Voyage 3 large performs best despite MTB bench results). Demonstrates using LLMs to synthesize realistic queries, then extracting summaries, clustering, and KPIs with the Cura library to find segments (e.g., ~40% data‑visualization traffic) and drive product decisions.
AI-written summary · about
chapters (5)
- 0:00Introduction to data analysis
- 1:32Evaluating retrieval inputs
- 7:12Analyzing conversation outputs
- 11:50Structuring and clustering data
- 16:31Takeaways and roadmap
related talks
32mReproducibility in Embedding Benchmarks
29mModel Evaluation for Custom Datasets With Open Models: Multi-Model Comparison With...
01h 30mBuilding a Search Engine
37mOptimal on Paper, Broken in Reality
24mLearning "Learning to Rank"
01h 08mGoing multi-modal
47mBuilding intelligent analytics at scale
32mMonitoring in the era of Generative AI, LLVMs, and embeddings