Democratizing NLP content modeling with transfer learning using GPUs
summary
Claims building in-house embeddings with using GPUs to enable transfer learning across tasks like recommendation, customer insights, and content-coverage measurement. Implements word2vec, character embeddings, skip‑thoughts and ELMo, aggregates layers with tf-idf weighting, then trains simple classifiers (logistic regression) on small label sets (often ~100–300 per class) augmented by weak supervision. Produces a queriable knowledge base stored in Redshift and reports >80% accuracy on many taxonomy nodes; training uses GPUs while consumers need not.
AI-written summary · about
chapters (15)
- 0:00Introduction
- 3:57Chegg products and data
- 6:07Transfer learning for NLP
- 11:04Word embeddings
- 13:14Character embeddings
- 16:44Sentence embeddings
- 18:30ELMo embeddings
- 22:51Fine-tuning embeddings
- 26:19NLP results
- 28:52Knowledge base creation
- 32:25Extracting concepts
- 34:32SQL queries
- 35:34Small data benefits
- 36:38Future of embeddings
- 39:38Q&A
related talks
45mDeep Learning for Natural Language Processing
41mBetter Machine Learning with Less Data
45mTransfer Learning - Entering a new era in NLP
31mTaking AI/ML to the next level: Text!
50mDemocratizing AI - Back-fitting end-to-end machine learning
46mWhy data-driven methods will shape the future (...)
01h 30mIntroduction to Deep Learning & Natural Language Processing
16mEmpower AI: Building Smarter Solutions with Embeddings