Taking AI/ML to the next level: Text!
summary
Builds a concrete pipeline that parses documents with Tika, converts abstracts into TF‑IDF vectors, and uses scikit-learn for a quick classifier and k‑means clustering over six years (hundreds) of Berlin Buzzwords talks. Migrates to embeddings (GloVe/word2vec via Apache MXNet) to compress 40k sparse terms into dense 250‑dim vectors and capture semantics, improving retrieval of semantically similar talks. Notes ElasticSearch wins on huge corpora, warns about memory, hyperparameter tuning, data bias and the need for regression tests.
AI-written summary · about
chapters (14)
- 0:00Introduction to AI/ML and text
- 1:31AI/ML buzzword and history
- 4:00Challenges with image classification
- 6:48Why text is different from images
- 9:54Pre-processing text data
- 11:57Turning text into numbers
- 16:08Building a classification model
- 18:34Clustering similar talks
- 22:04Advanced techniques with embeddings
- 24:31Hyperparameters and data tuning
- 26:15Bias and ethical considerations
- 27:37Measuring model performance
- 28:41Elasticsearch vs ML for text
- 29:44Key takeaways and next steps
related talks
42mThe Machine Understands Text — Learning and Search at Micro and Macro Scale
42mResearch in the Age of AI
30mExtracting insights from unstructured data
35mHow does text become data?
44mTeaching Computers ABCs: A Quick Intro to Natural Language Processing
47mThe Neural Search Frontier #bbuzz
01h 08mGoing multi-modal
32mGenerative AI Text and Multimodal Embedding Models for Real World Use Cases