Model Freshness and Its Infra Implications
summary
Meta recounts concrete infra changes that cut model update time: moving snapshot publishing into training, aggressive snapshot compression, and a network-local P2P forest for distribution. The system handles 20,000 models, distributed training with tens–hundreds of trainers, and distributed inference across partitioned hosts. Streaming deltas replace bulk syncs, achieving training-to-serve latency 10–30 minutes, with plans to target seconds while managing safety via tiered validation, sanity checks, and fast rollbacks.
AI-written summary · about
chapters (7)
- 0:00Model freshness intro
- 2:31Model freshness challenges
- 5:24From snapshots to streaming
- 11:10Real-time model updates
- 14:20Scaling training and inference
- 17:31Speed versus reliability
- 19:18Future of model freshness
related talks
29mAutomated Model Update & Evaluation
21mOperationalizing ML Training Infra at Meta Scale
31mApplied MLOps to Maintain Model Freshness on Kubernetes
12mQuickUpdate: a Real-Time Personalization System for Large-Scale Recommendation Models
42mImproving Machine Learning Development Reliability
33mDealing with the versioning of production-ready models
33mRuntime Modifications of Distributed Big Data AI Models and Their Parameters
51mSpeeding up deployment of machine learning models