Accelerating Neural Recommendation Training with Embedding Scheduling
Talkat 21st USENIX Symposium on Networked Systems Design and Implementation (NSDI '24) · 2024-04-16..04-18·
summary
A runtime embedding scheduler called Herald decides where embeddings are trained and which to sync, using an adaptive location-aware inputs allocator and an optimal communication-plan generator. It avoids unnecessary pulls and does on‑demand pushes, preserves BSP accuracy, and experimentally cuts embedding transmissions by 48%–89%. End‑to‑end training runs up to 2.11× faster over TCP and 1.61× over RDMA on 100 Gbps, evaluated on multi‑GPU clusters (10 servers, eight GPUs each) with real DLRM workloads.
AI-written summary · about
chapters (7)
- 0:00Embedding scheduling intro
- 2:47Distributed embedding challenges
- 4:54Embedding permutation opportunities
- 7:47Scheduling principles and real-time issues
- 9:57Herold's adaptive design
- 13:06Evaluation results
- 14:32Conclusion
related talks
20mOPER: Optimality-Guided Embedding Table Parallelization for Large-scale...
15mAdaEmbed: Adaptive Embedding for Large-Scale Recommendation Models
36mAccelerating AI training fleets with sched_ext
19mDynaRL: Flexible and Dynamic Scheduling of Large-Scale Reinforcement Learning Training
22mHypeReca: Distributed Heterogeneous In-Memory Embedding Database for Training...
08mTorchRec: A PyTorch domain library for large-scale recommender systems
20mBoosting Training and Inference Performance via Topology-Aware Scheduling of Heterogeneous Resources
13mTraining Embedding Model Resiliently for Multimodal M...