Releasing Ceph - Deep Dive Into Build Infrastructure
summary
A release pipeline migrates from monolithic per-distro servers and exposed GPG keys to automated ephemeral build nodes and a restricted signing station, replaces ad hoc bash with Jenkins Job Builder configs in a GitHub repo, and publishes status through the Shaman API and dashboard. Elastic pools auto-scale so the system hits 500 repos per day, cutting repo turnaround from ~36 hours to ~45–90 minutes. ARM builds run on donated physical hosts after OpenStack virtualization proves unreliable and community integration uses message buses and CentOS community builders.
AI-written summary · about
chapters (17)
- 0:00Intro
- 1:00What went wrong
- 4:34What went right
- 5:25The Jenkins Server
- 7:00Simple Matter of Programming
- 8:36Old Build Infrastructure
- 11:47Jenkins Job Builder
- 13:40Shaman
- 14:21API
- 15:00Automatic Server Pools
- 15:45Automatic Repository Production
- 16:30Scaling
- 18:55Balancing responsibilities
- 19:42Ecosystem expansion
- 21:11ARM builds
- 22:43Artifacts
- 28:28Lessons Learned
related talks
38mSIG Release - Deep Dive: Release Engineering Subproject
25mContainer pipeline for devs and enterprises alike!
30mCBS: A Containerized, Declarative, Flexible Build System Enhanced for Ceph Builds
37mWhat's (in) a Build System?
31mRelease Engineering as a Service - The Art of Integrating with K8s
43mDevOps for Mere Mortals
29mCeph and the future of Open Source Storage
13mProviding Continuous Integration at CERN