Tensorflow, deep learning and modern RNN architectures, without a PhD
summary
Walks through building working systems with concrete choices: TensorFlow dynamic_rnn plus GRU cells, bi‑directional RNNs, and GloVe embeddings; attention is implemented as a tiny two‑layer scorer so each timestep gets a learned weight. Demonstrates a toxicity detector trained on public comment data in minutes that highlights offending tokens and catches non‑curse threats, and a sequence‑to‑sequence translator using beam search, sample_softmax_loss and an attention mechanism that boosts long‑sentence quality — implemented with about a dozen lines via the seq2seq API.
AI-written summary · about
chapters (24)
- 0:00Intro
- 1:48RNN cell
- 4:32Training
- 5:17Replication
- 6:23Multiple layers
- 6:47Recurrent neural networks
- 7:34Recurrent cell architectures
- 8:42Language model
- 9:35Shakespeare
- 10:13Perspective
- 11:06Embed encode
- 13:27Embed encode properties
- 16:26Classification
- 17:15Bidirectional networks
- 20:17Building a toxicity detector
- 22:46Embedding
- 23:18Encoding
- 23:59Bidirectional
- 25:33Toxicity
- 28:12Google Translate
- 34:07Adding attention
- 38:45Embedding words
- 41:39Demonstration
- 42:45Sequence to Sequence API
related talks
26mUsing deep learning in natural language processing
01h 59mDeep Learning and Modern NLP
01h 30mIntroduction to Deep Learning & Natural Language Processing
42mAttention and Memory in Deep Learning Networks
36mNeural Networks
01h 26mFrom RNNs to Transformers: A Journey through Language Modeling
26mA Shallow Introduction to Self-Attention
01h 40mA Deep Dive into NLP with PyTorch