Modelling a text corpus using Deep Boltzmann Machines
summary
Builds probabilistic text embeddings from about 10,000 abstracts with a vocabulary of 1,000 using deep Boltzmann machines implemented in Python with numpy only. Uses a DBM with two layers of 50 units each, pretrained layerwise and trained with one-step Gibbs sampling and dropout 0.1. Produces document and word vectors whose clusters align with known brain activation patterns and tasks so the pipeline runs on a laptop in a few days and the method demonstrably recovers meaningful topics.
AI-written summary · about
chapters (8)
- 0:00Introduction
- 1:37Motivation
- 8:37Restricted Boltzmann Machines
- 12:51Training restricted Boltzmann Machines
- 19:08Text modeling
- 21:38Deep Boltzmann Machines
- 24:50Results on neuroscience abstracts
- 29:08Q&A
related talks
32mModelling a text corpus using Deep Boltzmann Machines in python
31mTaking AI/ML to the next level: Text!
01h 30mIntroduction to Deep Learning & Natural Language Processing
47mNatural language modelling- BlazingText algorithm
57mTutorial on Deep Learning III
25mGenerative Text Modelling: Scratching the surface
35mHow does text become data?
01h 03mWord Embedding For Fun and Profit