Tricks, tips and topics in Text Analysis
summary
Shows an actual pipeline: use the spaCy pipeline to tokenize, POS‑tag, lemmatize and mark stopwords, build bigrams with gensim, make a gensim dictionary/corpus and run gensim LDA (300‑article lis background corpus), then TF‑IDF + truncated SVD + classifiers (GaussianNB, SVM) with scikit‑learn, and generate text with a Keras LSTM. Surprises: noisy stopwords like “though/pron” skew topics; LDA runs in seconds on the small corpus; the LSTM produces readable but rough newspaper‑style text after brief training.
AI-written summary · about
chapters (16)
- 0:00Introduction
- 1:26Avoid NLTK, choose Spacy
- 9:16Cleaning text in Spacy
- 11:02Spacy pipeline and annotations
- 29:27Building a corpus with Spacy
- 34:47Pre-processing and corpus creation
- 35:49Bigram extraction
- 38:54Dictionary to bag-of-words
- 40:41Topic modeling with Jensen
- 47:01Refining topic models
- 50:16Topic modeling limits
- 54:35Text classification with scikit-learn
- 1:00:26Jensen tools beyond topic modeling
- 1:01:08Word embeddings and meaning
- 1:12:12Generating text with neural nets
- 1:26:35Summary
related talks
48mTopic Modelling (and more) with NLP framework Gensim
01h 38mNavigating the World of NLP and Text Analytics
30mKick-Start your Understanding of Machine Learning with Python
35mHow does text become data?
25mInventing Curriculum using Python and spaCy
43mGetting started with AI for free
01h 09mAutomatic Tagging of Short Texts with scikit-learn and NLTK
01h 01mWorkshop: Intro to topic modeling using Python