OPER: Optimality-Guided Embedding Table Parallelization for Large-scale...
summary
Claims a near-optimal planner and system that cut end-to-end DLRM embedding lookup cost using a heuristic search plus a distributed shared memory runtime and a single CUDA kernel for fused local/remote lookups and a local aggregated sparse allreduce. Achieves 2.3× training and 4.0× inference speedups on average, with 2.8× lookup improvement in multi-node tests. Implements heuristic assess-and-memory partitioning, per-embedding placement with selective replication, and a unified embedding address space that reduces communication and kernel overhead.
AI-written summary · about
chapters (14)
- 0:00OPER introduction
- 1:12DL recommendation model
- 2:18Embedding table challenges
- 3:42Existing parallelism limits
- 6:36Fine-grained parallelism benefits
- 7:42Fine-grained challenges
- 8:47OPER systematic solution
- 9:30Performance modeling
- 11:36OPER partitioning algorithm
- 13:25Data parallelism strategies
- 15:11OPER system optimizations
- 17:20Sparse gradient reduction
- 18:27OPER performance evaluation
- 19:50Ablation study
related talks
15mAccelerating Neural Recommendation Training with Embedding Scheduling
15mAdaEmbed: Adaptive Embedding for Large-Scale Recommendation Models
15mUSHER: Holistic Interference Avoidance for Resource Optimized ML Inference
26mEfficiently serving LLMs at scale.
16mWLB-LLM: Workload-Balanced 4D Parallelism for Large Language Model Training
15mOptimus: Accelerating Large-Scale Multi-Modal LLM Training by Bubble Exploitation
22mHypeReca: Distributed Heterogeneous In-Memory Embedding Database for Training...
20mSGLang: An Efficient Open-Source Framework for Large-Scale LLM Serving