Tensor Methods for Learning Latent Variable Models: Theory and Practice
summary
Shows that method-of-moments using third-order tensor moments plus whitening via rank-k SVD gives consistent, polynomial-time recovery of latent-variable parameters (Dirichlet topic models, mixed‑membership communities, HMMs, ICA), with sample complexity often near optimal (community learning needs n ~ K^2; overcomplete tensors recoverable up to rank ≲ D^{1.5}). Implements stochastic tensor power iterations on GPU, scales to ~1M‑node graphs, runs much faster than variational EM while matching or beating its accuracy.
AI-written summary · about
chapters (25)
- 0:00Intro
- 0:52Challenges in Unsupervised Learning
- 3:01How to model hidden effects?
- 4:29Moment Based Approaches
- 7:00Outline
- 8:43Beyond SVD: Spectral Methods on Tensors
- 9:40Spectral Decomposition
- 12:04Decomposition of Orthogonal Tensors
- 14:07Using Whitening to Obtain Orthogonal Tensor
- 14:53Putting it together
- 19:09Topic Modeling
- 19:51Geometric Picture for Topic Models
- 23:39Moments for Single Topic Models
- 25:01Moments under LDA
- 26:31Network Community Models
- 28:44Subgraph Counts as Graph Moments
- 32:08Multi-view Representation
- 36:50Main Results (Contd)
- 38:31Computational Complexity (k )
- 40:20Scaling Of The Stochastic Iterations
- 41:46Summary of Results
- 43:10Experimental Results on Yelp
- 46:19Beyond Orthogonal Tensor Decomposition
- 48:12Global Convergence k = Old
- 49:35Conclusion
related talks
53mTensor Decompositions for Learning Latent Variable Models II
54mLearning Overcomplete Latent Variable Models through Tensor Power Method
56mTensor Decompositions for Estimating Latent Variable Models
56mTensor Decompositions for Learning Latent Variable Models I
01h 03mRecent Progress in High-Dimensional Learning
52mLearning Mixture of Gaussians using Moment Tensor
33mAnimashree Anandkumar, Electrical Engineering and CS Dept, UC Irvine @ MLconf SEA
22mAnima Anandkumar, Electrical Engineering and CS Dept, UC Irvine