Generative Text Modelling: Scratching the surface
summary
Overview gives concrete results not hype it compares 3-layer LSTM character model trained on 16 MB Polish Wikipedia on Google Colab producing readable Polish fragments by iteration 45 and character-name patterns by 400–600, cites transformers like GPT-2 and Uber plug and play LM, notes Hugging Face transformers and distilled models, reports modern models up to 1.2 billion parameters and training costs around $10,000, and argues autogenerated fake news is plausible but not imminent based on quality and controllability metrics.
AI-written summary · about
chapters (14)
- 0:00Generative text modelling intro
- 1:20Language models in everyday life
- 3:08Measuring LM quality
- 5:40Template-based text gen
- 7:07Markov models for text
- 8:31Recurrent neural networks for text
- 11:01GANs for text generation
- 13:57Transformers revolution
- 17:16Fake news with GPT-2
- 18:20Plug-and-play text control
- 19:23Demo: simple LM
- 21:52Hugging Face use cases
- 23:36Training large models costs
- 23:59Key takeaways
related talks
45mTips For Detecting The Use Of AI-Generated Text
01h 00mInside GPT – Large Language Models Demystified
01h 02mAdvancing Diffusion Models for Text Generation
52mUnleash the Power of Large Language Models (LLMs)
43mBeyond the Hype: A Realistic Look at Large Language Models
45mHow Generative AI Works: LLMs and RAG Under the Hood
52mTowards fast and coherent long-form text generation
19mLet's Talk GenAI