Generalization in Diffusion Models Arises from Geometry-Adaptive Harmonic Rrepresentations
summary
Shows that diffusion-model denoisers trained on datasets from 1 to 100,000 images transition from memorization to generalization around that scale, with two independently sampled 100k-trained models mapping the same Gaussian seed to the same image. Uses Jacobian eigenanalysis to characterize denoisers as geometry-adaptive harmonic bases (GAHBs), validates on Cα and disk manifolds (optimal vs suboptimal cases), and implements a multiscale wavelet conditional denoiser that generates 320×320 images with 1 million parameters by enforcing a Markov in wavelet domain locality assumption.
AI-written summary · about
chapters (8)
- 0:00Introduction and motivation
- 1:51Memorization vs generalization
- 4:40Diffusion model mechanics
- 10:30Generalization experiments
- 22:42Inductive biases of denoisers
- 30:31Testing the GAHB hypothesis
- 40:24Scaling and receptive fields
- 51:04Multiscale wavelet denoisers
related talks
01h 21mGenerative AI with Diffusion Models
41mBuilding Generative Image & Video models at Scale
01h 09mLearning Generative Models from Corrupted Data
30mHow to train a large diffusion model
10mDiffusion models
49mGenerative Diffusion Models: Development, Training and Releasing Kandinsky 2.1
12mEfficient Workflow Serving for Diffusion Models with Many Adapters
01h 20mThe elusive generalization: classical bounds to double descent to grokking