Generative language models
summary
Overview of the evolution of natural language processing from early probabilistic models to modern large language models, discussing notable architectures and directions for future development.
AI-written summary · about
chapters (31)
- 0:00Introduction to language models
- 1:31Classical NLP and transformers
- 3:37GPT and decoder-only models
- 5:45Training approaches like FLAN
- 7:11Reinforcement learning for models
- 8:58Multimodal models like GPT-4
- 10:02Open-source large language models
- 11:27Saving resources with adapters
- 13:14Quantization for efficiency
- 15:23Optimizing attention mechanisms
- 17:12Gradient accumulation techniques
- 18:57Mixed precision training
- 20:28Parallelism in training
- 21:30Practical applications of models
- 22:57Creative tasks and scientific use
- 24:26Multimodal architectures
- 25:31Long sequence processing
- 26:13Notable models like Llama 2
- 27:19Mixture of experts concept
- 28:01Neural network innovations
- 29:02Future development paths
- 32:35Q&A session introduction
- 33:19Predictions about GPT-5
- 34:23Home-use language models
- 35:04Models in real business products
- 37:18Encoder vs decoder architectures
- 38:42Low-power system trends
- 41:11Multimodal models and robotics
- 45:03Symbiosis of models and algorithms
- 47:12Open-source vs proprietary models
- 49:22Closing remarks and Q&A
related talks
44mBuilding projects with language models
51mTowards a Pragmatic Understanding of Large Language Models
36mLarge Language Models for Real-World Applications - A Gentle Intro
15mFuture with Large Language Models
01h 11mThe Move Toward AGI: Why Large Language Models Surprised Almost Everyone... | Theoretically Speaking
40mThe Future of Language Models
30mLarge Language Models and Prompt Engineering for Developers
45mHow Generative AI Works: LLMs and RAG Under the Hood