AMI Creation And IaC Deploys via Semaphore
summary
Confluent automates AMI builds and Semaphore agent deploys using Packer, an open-source Agent AWS Stack (CDK) from Rendered Text and Terraform for dashboards, with seven AMIs and 36 agent pools managed. AMI builds take 30–60 minutes unless skipped by md5 hashing of sorted artifacts; pipelines run build → pre-prod → prod, with non-prod auto-deploy on master and manual prod promotions. PRs generate test AMIs, tests run on freshly deployed agents, and deployments flip ASG targets so existing jobs are unaffected. Testing depth and ephemeral non-prod pools remain improvement areas.
AI-written summary · about
chapters (10)
- 0:00Introduction
- 1:30Problem with manual AMI management
- 3:39Current tech stack overview
- 4:43Solution approach principles
- 5:25Semaphore pipeline design
- 6:52AMI creation optimizations
- 7:36Pipeline promotion process
- 9:42Infrastructure as code testing
- 11:52Results and improvements
- 12:54Future enhancements
related talks
11mAutomating AMIs for reliability, security, and cost (PRT074)
22mLeveraging Chef, Packer, and Habitat for Effortless AMI Deployments
16mDelivering Golden AMIs with Spinnaker
56mHCP Packer Learn Lab: Build a Golden Image Pipeline
28mUsing an Image Release Process for Security Wins
35mCustom CentOS AMI builds on AWS
06mIgnite - Blue Green Deployments Using Terraform + Packer + Jenkins
43mImmutable AWS Deployments with Packer and Jenkins