How well does diffusion model generate? - ...
summary
Provides the first end-to-end quantitative bound that couples training and sampling for denoising diffusion models, proving a non-asymptotic bound on gradient-descent training of wide ReLU nets and propagating that score error through discretized backward SDEs to a KL-style generation error. Delivers hyperparameter rules: a bell-shaped total weighting matches EDM practice, and polynomial (EDM) vs exponential schedules trade off when networks are poorly vs well trained. Analysis holds in an overparameterized NTK regime and leaves empirical vs population gaps.
AI-written summary · about
chapters (6)
- 0:00Introduction to diffusion
- 2:39Training and score matching
- 10:10Sampling and inference analysis
- 26:59Training dynamics and convergence
- 44:15Full error analysis and schedules
- 50:58Summary and future research
related talks
19mYou Might Not Need 50 Diffusion Steps
30mHow to train a large diffusion model
30mRepetita Non Iuvant: Why Generative AI Models Cannot Feed Themselves
49mGenerative Diffusion Models: Development, Training and Releasing Kandinsky 2.1
40mDiffusion Large Language Models
42mA Primal-dual Analysis of Margin Maximization by Steepest Descent Methods
01h 02mAdvancing Diffusion Models for Text Generation
01h 33mOn large language models and transformers: perspectives from physics, neuroscience, and theory