Making Deployments Safe at Meta
summary
Meta runs tens of thousands of changes every day for over 3.2 billion users and now gates rollouts with phased regional pushes, staging, and a set of default health checks that combine multiple metrics. A unified signals pipeline labels failures (false positives vs true regressions), feeds dashboards for backtesting metric/threshold changes, and scopes metrics via dimensional conversion to avoid noisy stops. Planned additions include a metric criticality catalog and a self-tuning change safety system to auto-configure precision/recall tradeoffs.
AI-written summary · about
chapters (5)
- 0:00Introduction to change safety
- 3:13Operational safety mechanisms
- 10:21Improving health check quality
- 13:23Tooling and back testing
- 15:29Future outlook and conclusion
related talks
18mBuilding Resilient Monitoring at Meta
21mYour Deployments Are Lying. AI Knows
20mConfiguration Safety at Scale with Ads
42mImproving Machine Learning Development Reliability
43mTo make mistakes is human. To auto-deploy it is DevOps.
24mContinuous Deployment at Scale
21mQuality Gates in Production: How We Turn OpenTelemetry...
46mProgressive Deployments with Consul Service Mesh