Sharding Clouddriver with Stormdriver
summary
Sharding layer routes API calls to multiple Clouddriver instances so accounts can live on separate processes instead of one huge JVM, enabling mixing Java and Go Clouddriver. Tests show a typical deployment with four Clouddrivers (four sets of ~100 AWS accounts plus a 100-namespace Kubernetes instance) uses about 200 MB RAM for Stormdriver and adds only ~3 ms to credential queries; Jaeger traces show an 11.75 ms end-to-end request with an 11.42 ms longest subcall. It polls credentials every 10s, limits list results to 2000 items, rejects unknown mutating requests, and is on github.com/opsmx/stormdriver (Apache-2.0).
AI-written summary · about
chapters (10)
- 0:00Introduction and about me
- 1:03What is Clouddriver
- 2:50Challenges with scaling Clouddriver
- 3:53How Stormdriver solves it
- 6:42Stormdriver routing mechanism
- 8:51Handling different request types
- 10:36Security and supported providers
- 11:43Advantages of Go Clouddriver
- 12:45Findings and deployment
- 13:06Performance demo and traces
related talks
18mStormdriver One Year Later
29mScaling Spinnaker for multi-region multi-cloud SaaS deployments
38mHighload — An Unexpected Journey
50mWeathering the Cloud Storm: Building Resilient Geo-Distributed Apps
57mLast- und Performancetests in der Cloud?!
28mFlexible Cloud, Whatever The Workload
31mScaling Autonomous-Driving R&D: A 100 PB, 100 Billion-File Open-Source Elastic File...
30mHow Workload Rightsizing Increases Availability