CI/CD Agility and Controlling Pipeline Sprawl

on YouTube ↗31m

" summary

Recommends right-sizing compute nodes to cut runtime and cost by using higher resource classes, runs unrelated steps in parallel with concurrent jobs to shrink build duration, caches selectively and updates only changed packages, and uses conditional tests instead of running all suites. Notes most setups use a one-to-one repo-to-config YAML pattern that causes sprawl, advocates extracting reusable scripts into pipeline libraries and cites CircleCI’s Orbs product as an implemented registry/versioning solution that centralizes logic and reduces vendor lock‑in.

AI-written summary · about

" chapters (33)

1. 0:00CI/CD Agility and Controlling Pipeline Sprawl Angel Rivera , Circleci
2. 0:46ci/cd : agility
3. 1:42CI/CD Tooling Obstacles
4. 3:43developer relations experiences: conversations community
5. 4:08Managing Pipelines at Scale
6. 6:15Resource Classes
7. 7:22Costs more but Executes Faster
8. 8:58Concurrent Jobs
9. 9:56Avoid Build Job Dependencies
10. 12:44Update Changed Dependencies
11. 13:44Selective Job Processing
12. 14:47Conditional Job Execution
13. 18:51pipeline : config: file
14. 19:141:1 ratio 1 x code repo = 1 x pipeline config
15. 20:02yaml is not a programming language
16. 20:25yaml is a data structure
17. 21:19syntax:growth : complexity
18. 21:39hard : navigate : manage
19. 21:57config: repo: sprawl
20. 22:24difficult : to: maintain
21. 22:40config: repo: dilema
22. 23:01options?
23. 23:11separation of concerns
24. 23:33common : pipeline : patterns
25. 23:44create : abstractions
26. 24:06parameterized : reusable: modules
27. 24:50Hard don't hardcode commands Code
28. 25:34pipeline : libraries
29. 25:46name: Set 53 Bucket Environment Variable command: export JAR_FILE="build/app_v10_2.jar export AWS_53_BUCKET="s3://bucket/current/
30. 26:52101 101 1101 centrally: managed: functionality
31. 27:43adds portability: minimizes vendor lock-in
32. 29:27recap: Pipeline Sprawl
33. 29:40identify common patterns in pipelines encapsulate functionality into parameterized scripts • use scripts to consolidate + centrally maintain code • scripts help to minimize vendor lock-in adds

" related talks

• 47mCI/CD Agility and Controlling Pipeline SprawlAngel Rivera · sfday 2020
• 33mCI/CD Agility and Controlling Pipeline Sprawl OutlineAngel Rivera · Indy Cloud Conf 2020
• 20mCI/CD Agility and Controlling Pipeline SprawlAngel Rivera · DevOpsDays NYC 2020
• 26mFaster, Cheaper, Leaner: Horizontally Scaling a CI PipelineMichal Cichra · FOSSASIA Summit 2019
• 32mPractical Tips & Tricks for CI/CD SuccessZan Markan · GOTO Amsterdam 2022
• 30mCI/CD Panel (Teams & Tools)Angel Rivera, Dawn Parzych, Vincent Demeester, Eric Sorenson · Continuous Delivery Summit 2019 - San Diego
• 46mThe Need for Speed (in Your CI/CD Setup)Zan Markan · DevOps Pro Europe 2021
• 27mOptimizing CI/CD Pipelines for Faster Time to MarketDavid Ayala · DevOpsDays Medellín 2024