Generative Models
summary
Models where labels are generated top-down on a tree get concrete statements: the optimal decoder is computed by belief propagation, phylogenetic algorithms can reconstruct the hidden tree in polytime, and a semi‑supervised procedure labels every leaf when the second eigenvalue passes the Kesten–Stigum threshold (2θ^2>1) for large alphabets. Weak bounded‑depth circuits (AC0) fail to beat random, threshold gates (TC0) can match BP when θ≈1, and general BP fits into NC1 showing why log-depth is sometimes necessary.
AI-written summary · about
chapters (17)
- 0:00Intro
- 0:51Why Deep Networks?
- 2:11Data Models and Deep Networks
- 3:06The Dream
- 6:58The Pure Theorist Model
- 11:19': A DL Theorist Perspective
- 13:41Hacker models
- 27:55Candidate 3: Scattering Transform
- 28:47The Question Remains
- 29:23Information Flow on Trees
- 31:06Is this process natural?
- 32:40What is the best classifier
- 34:48Provable Algorithms for learning classifier
- 37:06Depth Lower bounds
- 55:36Phylogenetic Reconstruction
- 56:51A semi supervised setting
- 58:39Deep Algorithms
related talks
01h 22mBeyond Decoding: Meta-Generation Algorithms for Large Language Models
01h 05mLogic and Probabilistic Circuits 1
44mChoose the right neural generative model for your problem
01h 03mLogic and Probabilistic Circuits 2
26mGenerative AI: Get a Grip … and a Look Under the Hood
12mDistilling models using the linear representation hypothesis
45mHow Generative AI Works: LLMs and RAG Under the Hood
52mGeneralization in the representations and computations of frontier language models.