Optimization on Manifolds

on YouTube ↗01h 06m

" summary

Overview of practical Riemannian optimization: definitions of manifolds (Stiefel, Grassmann, fixed-rank, SPD, hyperbolic), explicit formulas for tangent spaces, Riemannian metric, gradients and Hessians for embedded submanifolds, and computational tools that reduce to matrix factorizations. Introduces Manopt toolboxes, uses retraction maps (project or normalize or truncated SVD) to move on manifold, derives Taylor expansion of f∘retraction, and gives complexity for Riemannian gradient descent matching Euclidean rates O(1/√K).

AI-written summary · about

" chapters (16)

1. 0:00Start
2. 4:59Romanian Manifolds
3. 5:23What Is a Manifold
4. 8:23The Stifle Angle
5. 10:35Grass Man Manifold
6. 16:32What Is the Manifold
7. 20:22Why Do We Care about Manifolds
8. 20:54Tangent Vector
9. 32:34Metric Projection
10. 33:53The Tangent Bundle
11. 39:02A Vector Field on a Manifold
12. 47:37Hessians
13. 50:01Affine Connection
14. 52:23An Algorithm on a Manifold
15. 52:39Example of an Algorithm
16. 55:07Proving Global Convergence Rates

" related talks

• 11mGeometric Methods for Machine Learning and OptimizationMelanie Weber · Meet the Fellows Welcome Event
• 01h 18mOptimization IBen Recht · Big Data Boot Camp 2013
• 31mRandomized Riemannian Preconditioning for Quadratically Constrained ProblemsHaim Avron · Randomized Numerical Linear Algebra and Applications 2018
• 51mFirst-Order Methods and Hyperbolic ProgrammingJames Renegar · Hyperbolic Polynomials and Hyperbolic Programming
• 01h 01mRobustness, Stochastics, Uncertainty 2Sasha Rakhlin, Ben Recht, Laurent El Ghaoui · Bridging Continuous and Discrete Optimization Boot Camp 2017
• 49mMetric Representations: Algorithms and GeometryAnna Gilbert · Algorithmic Advances for Statistical Inference with Combinatorial Structure
• 46mSmooth Nonconvex Min-Max OptimizationMeisam Razaviyayn · Adversarial Approaches in Machine Learning 2022
• 50mUsing Dependent Types in Mathematical Optimization AlgorithmsRyan Orendorff · LambdaConf 2019