Tutorial: Methods from Statistical Physics II

on YouTube ↗01h 06m

" summary

Dense rank‑one Gaussian model (spiked Wigner): observations Y = (λ/n) x0 x0^T + W, x0 i.i.d. prior. Bayes free energy obeys a replica symmetric formula expressed via a one‑dimensional denoising free energy; derivation uses the cavity method / leave‑one‑out. Algorithms: approximate message passing (AMP) iterations are provably tracked by state evolution, reducing to a scalar fixed point Q. Spectral and AMP thresholds sit at λ=1; sparse PCA can create a discontinuous (information‑algorithm) gap where AMP fails below the Bayes optimal regime.

AI-written summary · about

" chapters (18)

1. 0:00Start
2. 0:10Approximate Message Passing
3. 5:37Bayes Rule
4. 6:57The Replica Symmetric Formula
5. 7:15Vrs of Lambda
6. 10:36The Cavity Method
7. 11:09Landmine Analysis
8. 13:34Heuristic Assumptions
9. 23:54Replica Symmetric Hypothesis
10. 24:42Gaussian Process
11. 34:13Query Interpolation
12. 35:13Additive Gaussian Model
13. 40:19Compute Marginals
14. 41:54State Evolution
15. 52:05Posterior Mean
16. 1:02:25Sparse Pca
17. 1:03:08Discontinuous Phase Transition
18. 1:04:48Discontinuous Phase Transitions

" related talks

• 45mTAP Free Energy, Spin Glasses, and Variational InferenceZhou Fan · Robust and High-Dimensional Statistics (Simons Institute workshop, 2018)
• 49mApproximate Message Passing Algorithms For Orthogonally Invariant ModelsZhou Fan · Algorithmic Advances for Statistical Inference with Combinatorial Structure
• 36mTo Be (Hard) or Not Be:...Florent Krzakala · Rigorous Evidence for Information-Computation Trade-offs
• 01h 04mStatistical Physics and Computation in High DimensionFlorent Krzakala, Lenka Zdeborova · Probability, Geometry, and Computation in High Dimensions Boot Camp (Simons Institute, Fall 2020)
• 49mAnalysis of Algorithms on Dense Matrices using Approximate Message PassingAndrea Montanari · Random Instances and Phase Transitions (Simons Institute, 2016)
• 01h 04mNonparametric Bayesian Methods: Models, Algorithms, and Applications IIMichael Jordan · Foundations of Machine Learning Boot Camp 2017
• 01h 11mStatistical Physics and Computation in High DimensionFlorent Krzakala, Lenka Zdeborova · Probability, Geometry, and Computation in High Dimensions Boot Camp (Simons Institute, Fall 2020)
• 35mPhase Retrieval In High Dimensions: Statistical And Computational Phase TransitionsAntoine Maillard · Rigorous Evidence for Information-Computation Trade-offs