Better Machine Learning with Less Data
summary
Argues that reusing pretrained embeddings and transfer learning yields workable models with far less labeled data: word-level embeddings like GloVe reach ~65% sentiment accuracy with ~50 examples, TF‑IDF sits near chance and needs ~10,000 examples, while fine‑tuned transformer models (ELMo / GPT / BERT-style) beat both. Shows training cost: a BERT‑scale run uses 16 cloud TPUs and takes ~4 days (~$12,000 per run). Demonstrations use sentiment and image benchmarks to prove fewer labels plus pretrained features get practical results.
AI-written summary · about
chapters (8)
- 0:00Introduction and agenda
- 3:23Why deep learning exploded
- 10:59Problems with traditional ML
- 15:33Deep learning limitations
- 21:13How embeddings work
- 28:52Training embedding models
- 36:57Modern embedding breakthroughs
- 39:24Key takeaways and conclusions
related talks
44mTransfer learning in NLP
32mAn Introduction to Transfer Learning in NLP and HuggingFace
40mPanel discussion "NLP"
45mTransfer Learning - Entering a new era in NLP
01h 26mBagging to BERT- A tour of applied NLP
13mTransfer Learning: Leveraging Pretrained Models with Limited Data
43mDemocratizing NLP content modeling with transfer learning using GPUs
43mThe Evolution of Domain Transfer in the Era of Pre-trained Language Models