A General Framework for Communication-Efficient Distributed...
summary
Proposes the CoCoA framework that plugs existing single‑machine solvers into a primal‑dual distributed loop, applies local updates immediately, and averages machine updates with a tunable aggregation so communication drops dramatically. ProxCoCoA later permits summing updates for more aggressive master steps and extends to L1 by switching to primal coordinate methods when data is feature‑sharded. Implements experiments in Spark on EC2, reports 50x speedup versus Spark MLlib baselines, with formal convergence guarantees and open‑source code (CoCoA framework, ProxCoCoA).
AI-written summary · about
chapters (12)
- 0:00Intro
- 0:41Machine Learning with Large Datasets
- 0:53Machine Learning Workflow
- 4:12Mini-batch Limitations
- 5:48Primal-Dual Framework
- 7:40Immediately Apply Updates
- 8:11Average over K
- 8:43COCOA-v1 Limitations
- 9:38L1 Regularization
- 10:29Solution: Solve Primal Directly
- 13:04COCOA: A Framework for Distributed Optimization
- 13:55Impact & Adoption
related talks
15mCocoon: A System Architecture for Differentially Private Training with Correlated Noises
01h 20mAn Introduction to Distributed, Massively Parallel, and Local Computation Algorithms
15mAutoCCL: Automated Collective Communication Tuning for Accelerating Distributed and...
28mCommunication Avoiding Algorithms for Linear Algebra and Beyond
19mSol: Fast Distributed Computation Over Slow Networks
21mMassive proofs for the masses
45mSymbiotic Relations between Decoupled Training, Optimization, and Federated Learning
30mThree Goals in Parallel Graph Computations: High Performance, High Productivity, and Reduced Comm...