Geometric Methods in Optimization and Sampling Boot Camp
2021-08-30..2021-09-03
-- online --
[Optimization, Statistics and Uncertainty, Probability, Geometry, and Computation in High Dimensions Boot Camp (Simons Institute, Fall 2020), Foundations of Data Science Boot Camp 2018, Optimization and Algorithm Design 2023, Semidefinite Optimization, Approximation and Applications 2014, Bridging Continuous and Discrete Optimization Boot Camp 2017]
who it's for
machine learning researchers, computational mathematicians, probabilists and stochastic modelers, statisticians working on inference, phd students in applied mathematics, numerical optimisation engineers
topics
Riemannian optimization on manifolds; geodesic convexity for matrix problems; proximal and projected mappings; cubic-regularized Newton methods; variance-reduced finite-sum algorithms (SVRG/SAGA); overdamped and underdamped Langevin dynamics; Hamiltonian Monte Carlo and symplectic integrators; Schrödinger bridge / diffusion bridges; optimal transport and entropic regularization (Sinkhorn); Wasserstein gradient flows and Benamou-Brenier
| title | author | length |
|---|---|---|
| Optimization Crash Course | Ashia Wilson | 42m |
Crash course gives concrete guarantees: first/second-order oracle models, step size ≤ 1/L for gradient steps, and a three-line Lyapunov-style proof that… | ||
| Optimization Crash Course (continued) | Ashia Wilson | 01h 07m |
Discrete acceleration closes the classic GD vs lower‑bound gap to a √κ dependence (Polyak/Nesterov ODE viewpoint and energy/Lyapunov proofs) but needs… | ||
| Optimization on Manifolds | Nicolas Boumal | 01h 06m |
Overview of practical Riemannian optimization: definitions of manifolds (Stiefel, Grassmann, fixed-rank, SPD, hyperbolic), explicit formulas for tangent… | ||
| Geodesic Convexity and Optimization | Suvrit Sra | 01h 12m |
Geodesic convexity explains why some nonconvex problems repeatedly return the same solution and gives concrete tools: on the PSD manifold with Fisher–Rao… | ||
| Sampling Crash Course | Andreas Eberle | 01h 03m |
Shows that random-walk Metropolis has exponentially small acceptance unless step size scales like O(1/d), uses conductance to turn that into provable lower… | ||
| Sampling Crash Course (continued) | Andreas Eberle | 01h 03m |
Shows that reflection coupling yields contraction only after applying a carefully chosen concave WF distance f with explicit constants depending on L, k, R… | ||
| Sampling Using Diffusion Processes, from Langevin to Schrödinger | Maxim Raginsky | 01h 14m |
Derives an explicit time‑dependent drift that exactly transports Brownian noise to a target law at t=1 by minimizing path‑space KL (the Schrodinger bridge)… | ||
| An Introduction to Hamiltonian Monte Carlo Method for Sampling | Nisheeth Vishnoi | 01h 10m |
Algorithm samples by drawing a Gaussian momentum, simulating Hamiltonian flow for time T, then refreshing momentum; it removes the Metropolis accept/reject in… | ||
| Crash Course on Optimal Transport (continued) | Mikaela Iacobelli | 55m |
Defines gradient flow on a Hilbert space by matching differential and gradient through a chosen inner product and shows the Dirichlet energy’s L2… | ||
| Computational Optimal Transport | Marco Cuturi | 01h 22m |
Computes that classic discrete OT is cubic (Hungarian / network‑flow), fragile (nonunique P with Jacobian zero or infinite) and statistically cursed in 5–50D… | ||
| Gradient Flows and Optimal Transport in Discrete and Quantum Systems | Jan Maas | 01h 19m |
Shows that Markov chain evolution equals the gradient flow of relative entropy for a new Riemannian transport metric built from a discrete Benamou–Brenier… | ||
| Crash Course on Optimal Transport | Mikaela Iacobelli | 01h 02m |
Covers Monge vs Kantorovich formulations and shows that Monge maps can be empty while Kantorovich couplings always exist, because Kantorovich turns transport… | ||
| Optimal Transport and PDE: Gradient Flows in the Wasserstein Metric (continued) | Katy Craig | 01h 04m |
Time discretization uses the implicit Euler/proximal map as a minimization problem and yields existence and uniqueness under lower semi continuity and… | ||
| Consensus-Based Interacting Particle Systems and Mean-field PDEs for Optimization and Sampling | José Antonio Carrillo | 01h 07m |
Introduces a consensus-based particle method that uses weights e^{-βF} to compute a weighted average of particle locations, derives a mean-field Fokker–Planck… | ||
| PDE on Graphs and Their Continuum Limits | Dejan Slepčev | 01h 10m |
Shows when graph PDEs actually approximate a continuum PDE: with kernel weights and ε→0 the graph Laplacian equals a nonlocal integral operator and then… | ||
| (Hidden) Symmetries of Computational Problems | Avi Wigderson, Michael Walter | 01h 07m |
Provides a polynomial-time algorithm for the non‑commutative singularity problem by converting it to an operator scaling instance and running an… | ||
| (Hidden) Symmetries of Computational Problems (continued) | Avi Wigderson, Michael Walter | 01h 11m |
Claims that geodesic convexity explains why greedy scaling (e.g., Sinkhorn) works, and that three problems — nullcone membership, norm minimization (capacity)… | ||
| Optimization and Sampling Under Symmetry: Part 1 -- Examples | Nisheeth Vishnoi | 01h 11m |
Starts from two concrete claims: minimizing vAv over the complex unit sphere equals the matrix's smallest eigenvalue by recasting rank‑one projections as a… | ||
| Optimization and Sampling Under Symmetry: Part 2 -- Lie Theory | Jonathan Leake | 01h 09m |
Lie theory turns the concrete claims into tools: minimizing a linear functional over a unitary conjugation orbit becomes adjoint orbit → convex polytope via a… | ||
| Optimal Transport and PDE: Gradient Flows in the Wasserstein Metric | Katy Craig | 59m |
Survey of specific PDEs that are Wasserstein gradient flows, the particle schemes used, and when they provably match the continuum: continuity equation with… | ||
| nothing matches | ||