Deep Learning for Machine Translation: a paradigm shift
summary
Claims that neural methods replace rule and SMT since 2016 and deliver much better fluency, shows the classic SMT pipeline: noisy channel framing, IBM alignment learned by expectation maximization and beam search decoder, then contrasts end-to-end neural machine translation using encoder‑decoder RNNs with attention to avoid the bottleneck. Demonstrates practical tricks: transfer learning by fine‑tuning last layers (ModernMT), token‑prefixing for zero‑shot multilingual transfer, and unsupervised NMT trained on monolingual corpora like Wikipedia (2017) that works without parallel data.
AI-written summary · about
chapters (21)
- 0:00Intro
- 3:00Statistical Machine Translation
- 4:11Noisy channel model
- 5:48Word choice and word reordering
- 7:52Translation model
- 8:12Computing alignments intuition
- 8:52Training Data
- 9:17Computing alignments: Expectation Maximization
- 11:27Translation options as a coverage set
- 12:10Decoding in action
- 14:40Neural machine translation
- 18:36Neural network review: feed-forward
- 20:47Recurrent network
- 21:54Neural language model
- 23:57Encoder - Decoder architecture
- 27:10Sequence-to-sequence (seq2seq) architecture
- 28:33Summary vector as information bottleneck
- 30:04Attention model
- 31:10Neural domain adaptation
- 33:08Zero shot translation: Google Neural MT
- 36:25Unsupervised NMT
related talks
40mHow Can Artificial Intelligence use Big Data for Translating Documents?
26mUsing deep learning in natural language processing
39mDeep Learning vs. Conventional Machine Learning
46mWhy data-driven methods will shape the future (...)
31mRemaining Challenges in Deep Learning based NLP
44mTransfer learning in NLP
50mSequential Attention-Based Neural Machine Translation
01h 26mFrom RNNs to Transformers: A Journey through Language Modeling