Evaluating Topic Models
summary
Shows which evaluation steps actually change decisions: uses pyLDAvis and gensim for visualization, computes coherence scores and word‑intrusion tests, and ships a companion notebook at github.com/mattilyra/pydataberlin-2017. Warns that perplexity misleads, that coherence depends on segmentation and reference corpus, that pyLDAvis’s bubble areas and 2‑D projections hide scale and projection artifacts, and that a 35‑topic model can look better than a 100‑topic one even when the larger model contains extra useful topics plus lots of junk.
AI-written summary · about
chapters (7)
- 0:00Evaluating topic models
- 1:50Eyeballing topic models
- 9:17Interactive topic exploration
- 16:27Intrinsic evaluation limits
- 26:25Human judgment methods
- 28:57Evaluation challenges
- 32:30Q&A
related talks
01h 01mWorkshop: Intro to topic modeling using Python
45mDecoding Topics: A Comparative Analysis of Python’s Leading Topic Modeling Libraries Using Climate C
28mBack to the Future: Going Beyond Worst Case Analysis
35mLeveraging LLM for text augmentation in Topic Modeling
10mThrough the Lens of LLMs: Unveiling Differential Privacy Challenges
19mHow to look at your data
01h 19mGenerating Customers Insights with Topic Modelling and HuggingFace SetFit Method
59mWhy Great Models Fail: Lessons From 9 Years of Deploying ML Models