Building distributed systems is hard - case study (...)
summary
GCE Autoscaler automatically resizes managed instance groups to keep target utilization (CPU, HTTP LB or arbitrary Stackdriver metric), honoring min/max and cooldown while treating zones independently. Implementation uses Paxos-based master election so one replica makes mutations, reads replicas build in-memory models, and it stores authoritative state in Spanner for global consistency. Backend ingests thousands of per-instance samples by keeping aggregated per-zone samples and degrades gracefully when a signal or component is missing. The service runs in production and is offered as part of Google Cloud.
AI-written summary · about
chapters (21)
- 0:00Introduction and speaker background
- 1:55Why distributed systems at Google
- 4:04Challenges of single data center approach
- 5:27Global distribution and replication
- 7:15Borg cluster management system
- 9:00Treating data centers as resources
- 10:48Automated resource estimation
- 11:53Scheduling policies and priorities
- 12:16Safe and gradual service releases
- 14:43Testing culture at Google
- 15:06Google Cloud and virtual machines
- 16:30Instance groups and autoscaling basics
- 18:40Configuring GC Autoscaler metrics
- 20:25Regions zones and fault tolerance
- 22:14Handling scale and replication
- 23:41Master election and consensus
- 25:07Inter-service communication
- 26:12Data consistency challenges
- 27:38Spanner global consistency database
- 28:00Handling component failures
- 29:07Summary and key takeaways
related talks
43mScalability Intro – Shyam Jeedigunta, Google & Bob Wise, AWS (Any Skill Level)
06mPlanet scale 2.0: Cloud games done right
14mMaximizing Aerospike Administration in GCP
33mGoogle Cloud Spanner: Global consistency at scale
36mCluster Autoscaler Evolution
28mScale Smarter Not Harder: How Extending Cluster Autoscaler Saves Millions
16mAutoscaling Distributed System with BOSH
42mCapacity vs Efficiency: Building a Globally Scalable Cloud Database