Analysing user comments with Doc2Vec and Machine Learning classification
summary
Doc2Vec training on scraped German comment threads yields concrete artifacts: Doc2Vec document embeddings, word vectors, and a linear classifier pipeline. Data volume is large — 280k / 170k / 50k comments — and preprocessing is minimal (lowercase, strip punctuation). The claim is measurable: a logistic/stochastic gradient classifier reaches about 50% test accuracy versus 33% random, with 60% training accuracy; prototypical comments expose clear class-specific patterns and extremist language.
AI-written summary · about
chapters (21)
- 0:00Introduction
- 1:49Comment quality across sites
- 3:17Data scraping and preprocessing
- 5:03Word2Vec and word embeddings
- 8:16How Word2Vec learns
- 14:22From Word2Vec to Doc2Vec
- 16:09Training Doc2Vec
- 17:13Analyzing word embeddings
- 19:20Word vector math
- 20:47Doc2Vec for classification
- 23:15Testing the hypothesis
- 25:26Recap and conclusions
- 26:38Q&A: Embedding space
- 27:21Q&A: Preprocessing and typos
- 28:03Q&A: Prototypical comments
- 28:41Q&A: Punctuation and features
- 29:25Q&A: Baseline comparisons
- 30:07Q&A: New comments to vectors
- 31:32Q&A: Linking to articles
- 33:19Q&A: Public API plans
- 34:13Q&A: TF-IDF comparison
related talks
31mTaking AI/ML to the next level: Text!
46mZero-Shot Classification with OpenAI and...
42mDocument Vectors in the Wild
44mContent Classification: NLP vs. AI in the Age of Data Vectorization
21mMaking Sense of Unstructured Threat Data
44mTeaching PHP new tricks with machine learning
31mDeep Learning Techniques for Natural Language Processing
30mProductionizing an unsupervised machine learning model to...