Control Theory
summary
Travis CI hits a hard cap of ~500 concurrent jobs so queued jobs spike into a red backlog; AWS Auto Scaling only offers metric, schedule or fixed-step policies and new workers take ~20 minutes to accept jobs. The team builds metric-driven and scheduled scaling via the AWS API and experiments with larger headroom (more always-on VMs) and historical prediction (schedule extra instances Monday 09:00 UTC); costs drop queue time but delays still cause surges to miss SLAs.
AI-written summary · about
