Primus: Unified Training System for Large-Scale Deep Learning Recommendation Models
summary
Primus implements a unified resource controller that standardizes YARN and Kubernetes semantics, runs dynamic horizontal+vertical scaling across multi‑cluster pools, and cuts cluster training cost by 17.1% at cluster level while raising CPU utilization 50%→80% per job. A three‑tier data definition plus a Data Task Graph generator handles HDFS/FStore, Kafka, Iceberg/Hoodie sources and yields 23× faster task generation and higher throughput versus Flink. The hybrid MTRM training paradigm mixes batch replay and stream priority, producing a 0.4%–2.4% advertising revenue lift in production.
AI-written summary · about
chapters (12)
- 0:00Primus training system
- 1:26Large-scale DRM training challenges
- 5:00Three key solutions
- 6:07Traditional vs Primus architecture
- 7:32Unified resource scheduling
- 8:16Unified data orchestration
- 9:21Unified MTRM training
- 10:25Dynamic scaling evaluation
- 12:57Data generation performance
- 14:20MTRM results and revenue
- 15:22Implementation lessons
- 16:04Summary and future
related talks
32mResurrecting a Recommendations Platform
20mFoundation Models for Big Data and Prompt Tuning
31mBuilding and Managing a Centralized ML Platform with Kubeflow at...
33mKubernetizing Big Data and ML Workloads at Uber
29mLarge Scale Distributed Deep Learning with Kubernetes Operators
46mMachine Learning with Kubernetes
12mQuickUpdate: a Real-Time Personalization System for Large-Scale Recommendation Models
13mPreview: ML Training