QuickUpdate: a Real-Time Personalization System for Large-Scale Recommendation Models

on YouTube ↗12m

" summary

Real-time personalization for terabyte-scale recommendation embeddings reduces published bandwidth by over 13x while matching a fully fresh model’s accuracy. The system selects rows by optimizer-state Δ pruning, publishes only top-changing embedding rows (e.g., 10% updates keep accuracy loss <0.01% after 10 hours), applies quantized intermittent full snapshots to repair drift, and avoids reshuffles by restricting selections to non-pruned rows. Updates are directly patched on serving GPUs rather than buffered, and results come from evaluation on a large production Meta recommendation model with real-world traffic.

AI-written summary · about

" chapters (12)

1. 0:00QuickUpdate introduction
2. 0:44Recommendation models at scale
3. 2:09Meta pipeline
4. 2:52Real-time updates importance
5. 3:55Frequent model publishing challenges
6. 4:37Pruning embedding tables
7. 6:21Handling pruning inconsistencies
8. 7:24Serving model updates
9. 8:06QuickUpdate workflow
10. 9:10QuickUpdate evaluation
11. 10:35Bandwidth savings
12. 11:38Conclusion and key takeaways

" related talks

• 27mEnhancing Personalization Using Recommendation Systems & Multi-Armed BanditsDmitriy M. · DSC Europe 24
• 20mDataEngBytes 2023 - PER-T1-06 - Kamal HossainKamal Hossain · DataEngBytes Perth 2023
• 15mAdaEmbed: Adaptive Embedding for Large-Scale Recommendation ModelsWei Zhang, Liu Rui, Jongsoo Park, Jie You, Mosharaf Chowdhury, Fan Lai, Zeliang Chen, Ellie Wen, Jianyu Huang, William Tsai, Xiaohan Wei, Yuxi Hu, Sabin Devkota, Xing Liu, Paul Rivera, Chun-cheng Jason Chen · OSDI 2023
• 18mBuilding an Incrementally Trained, Local Taste Aware, Global Deep Learned Recommender System ModelAnoop Deoras · MLconf SF 2019
• 33mPowering Real-time Recommendations with Machine Learning on GCPMatt Bushell, Vipul Naik · Aerospike Digital Summit 2021
• 24mLarge Scale On-Demand Low-Latency Near Real-Time PredictionsGabor Melli · Scale By The Bay 2019
• 39mGrowing user engagement with RL-driven personalisationChris Wilkin · PyData London 2024
• 16mCheck-N-Run: a Checkpointing System for Training Deep Learning Recommendation ModelsAssaf Eisenman, Kiran Kumar Matam, Steven Ingram, Dheevatsa Mudigere, Raghuraman Krishnamoorthi, Krishnakumar Nair, Misha Smelyanskiy, Murali Annavaram · 19th USENIX Symposium on Networked Systems Design and Implementation (NSDI '22)