Deploying Like a Boss: From 2 Servers to 20,000
summary
Starts with fragile tar + scp pushes and single-host git pulls, proves brittle past tens of servers, and moves to a pull-based coordinator with workers that pings every 30s (5s during deploys), backed by a Custo hardware DB for pools and policies; build distribution moves to tiered rsync/HDFS/BitTorrent patterns. At larger scale the stack integrates Mesos + Marathon for cluster scheduling, container/cgroup isolation and automated rollbacks, and an autoscaling loop driven by per-service CPU metrics (Graphite) that initially issues recommendations and later acts.
AI-written summary · about
chapters (24)
- 0:00Intro
- 0:50Agenda
- 1:30Who am I
- 3:04First webpage
- 4:40Deployment system
- 5:20Early deployment system
- 8:50Middle stage deployments
- 9:20Thirdparty libraries
- 10:46Pros and cons
- 13:02Build distribution
- 13:30Tiered Rsync
- 15:43Push vs Pull
- 18:38CUSTO
- 19:48Deploy
- 23:20Single Service
- 25:57Rollback
- 28:04The Future
- 28:16Quick Poll
- 28:46Cluster Management
- 35:11Integration
- 37:38Composition
- 38:11Scaling
- 40:59Mesos frameworks
- 43:27The long journey
related talks
45mLarge-Scaled Deploy Over 100 Servers in 3 Minutes
22mScalable Deployments...
31mContinuous Delivery Lessons Learned Through Hypergrowth
46m10+ Deploys Pe
52mDevops @Scale
24mContinuous Deployment at Scale
40mFrom Push to Pull: Managing Mutable Infrastructure at a Global Scale
43mServing for the win - Deploys and hosting for the rest of us