Tutorial: Methods from Statistical Physics I
summary
Introduces the Gaussian additive model with explicit posterior as a Gibbs measure and defines partition function Z and free energy F = E log Z; shows F′ and F″ give posterior moments and derives the I‑MMSE relation (MMSE from dF/dλ). Works through a Gaussian mean‑location example, finds a sharp phase transition at λc = 2 log 2 where MMSE jumps from 1 to 0, explains annealed vs quenched mismatch caused by rare large‑noise events and conditions them away. Mentions extension to spiked Wigner and AMP algorithms next.
AI-written summary · about
chapters (17)
- 0:00Start
- 2:48Gaussian Additive Model
- 5:01Partition Function
- 8:04Posterior Mean
- 9:28Bayes Rule
- 12:58Derivatives of the Free Energy
- 13:41Derivatives of F
- 15:55Gibbs Average
- 21:25Mutual Information
- 25:39The Imse Theorem
- 26:50Phase Transition
- 29:04Maximum Likelihood Estimator
- 30:51Compute the Free Energy
- 34:16Pyramid Analysis
- 42:06The Moments Method
- 42:55Conditional Expectation
- 57:37Spike Structure Model
related talks
01h 04mNonparametric Bayesian Methods: Models, Algorithms, and Applications II
45mTAP Free Energy, Spin Glasses, and Variational Inference
35mPhase Retrieval In High Dimensions: Statistical And Computational Phase Transitions
01h 02mNonparametric Bayesian Methods: Models, Algorithms, and Applications III
01h 06mNonparametric Bayesian Methods: Models, Algorithms, and Applications I
39mPhase Retrieval in High Dimensions: Statistical and Computational Phase Transitions
01h 04mStatistical Physics and Computation in High Dimension
46mSolvable Model of Unsupervised Feature Learning