Parameter Server on Flink, an approach for model-parallel machine learning
summary
A Parameter Server implemented on Flink Streaming API stores model shards on server operators and runs updates on worker operators, supporting model-parallel training for passive-aggressive text classification and matrix factorization recommendation. Implementation uses stream iterations, handles sparse pulls and sparse delta pushes, and runs asynchronously so streaming training often converges faster than a synchronized Flink batch SGD; a benchmark feeds 15 weeks of data in seven minutes. Known issues include loop deadlocks in Flink’s iterations API; workarounds are Kafka or manual backpressure control. Code and STREAMLINE project links are provided.
AI-written summary · about
chapters (12)
- 0:00Introduction and background
- 2:08Model parallel training explained
- 7:58Recommendation systems with matrix factorization
- 13:12Parameter server architecture overview
- 17:11Implementing passive-aggressive algorithm
- 21:11Matrix factorization on parameter server
- 24:33Parameter server in Flink API
- 26:45Technical challenges with loops API
- 29:15Batch vs streaming machine learning
- 32:08Asynchronous training results
- 34:43Combining offline and online learning
- 36:27Q&A and conclusion
related talks
05mSimplified distributed training with tf.distribute parameter servers
40mStreaming ML with Flink
18mMachine learning with Flink in Weibo
15mFeature Stores: Powering Machine Learning Infrastructure with Apache Flink
25mLifecycle of a Parameter
35mIntroducing Tensorflow on Flink
42mAlex Smola, Professor in the Machine Learning Department, Carnegie Mellon University
03mInterview with Márton Balassi