AutoScaling CI infrastructure on Google Container Engine
summary
Jenkins runs Dockerized Rails tests on GKE using parallel_tests to partition ~14,000 specs into 128 groups; a cold run launches 64 build pods across eight 8‑CPU nodes and each test group finishes in ~100–200 seconds. Infrastructure uses docker-compose style init containers, Google Container Registry for images and cheap preemptible instances (≈80% discount) to keep cost around €0.20–€0.25 per test thread; auto-scaling node pools spin up for pending pods and scale down after builds, and the setup meets the concurrency and cost goals.
AI-written summary · about
chapters (14)
- 0:00Intro
- 1:17PhraseApp
- 6:11Docker Compose
- 11:35Kubernetes
- 14:56Google Container Engine (GKE)
- 16:10Node Pools
- 17:02Google Container Registry
- 18:22Preemptible instances
- 19:33Prices per core/thread per day
- 21:12AutoScaling
- 23:55Build Container
- 24:57Infrastructure Container
- 25:52Test Execution
- 36:52Outlook
related talks
44mContainers and CI/CD on Google Cloud
35mJenkins Scaling and Organization for an Efficient CI
43mInfinite Space for Your Tests
38mUsing Containers for Continuous Integration and Continuous Delivery [I]
41mKubernetes in Action: Examples from Google Container Engine
40mKubernetes - Easy Peasy
28mWith Docker and k8's, this is not your grandparents' infra anymore!
38mChef + Google Kubernetes Engine