Remaining Challenges in Deep Learning based NLP
summary
Deep learning powers word2vec, RNNs and LSTMs but key gaps remain: models need massive domain-specific data (Google Translate defaults to Igbo/Bible defaulting when corpora are sparse), popular benchmarks contain generation artefacts (SNLI yields hypothesis-only bias), distributional embeddings conflate senses and fail on multiword noncompositional phrases (polysemy in word2vec), sentence vectors miss implicature and pragmatics, training is brittle to hyperparameters, and interpretability and fairness measures (Amazon hiring example) are still unreliable despite adversarial de-biasing.
AI-written summary · about
chapters (14)
- 0:00Introduction
- 1:04About me
- 1:33Popular media examples
- 2:05Adding more layers
- 3:15Representation learning
- 6:59Recurrent neural networks
- 8:24Big data problem
- 9:00Overfitting
- 10:54Life cycle
- 15:38Distributional similarity
- 18:15Sentence embedding
- 20:39Not robust enough
- 21:41Lack of interpretability
- 26:08Questions
related talks
55mKeynote - Natural Language Processing: Challenges and Next Frontiers
01h 30mIntroduction to Deep Learning & Natural Language Processing
49mDeep Learning Applied to Natural Language Processing (NLP)
31mDeep Learning Techniques for Natural Language Processing
17mNatural Language Processing in Automation
31mThe dark side of deep learning
01h 31mNatural Language Understanding: Foundations and State-of-the-Art
46mWhy data-driven methods will shape the future (...)