DevOps in a Data Warehouse: Inside Out
summary
A small ops team of four turns a legacy Hadoop cluster into an infrastructure-as-code platform, spins up repeatable AWS deployment pipelines, and exposes a common Jenkins service plus a container-as-a-service catalog so squads get disposable CI boxes and dedicated Docker hosts. Work shifts from reactive firefighting to a rotating Kanban model with a separate platform board, reducing unplanned tickets, keeping the backlog stable, and delivering the promised data products and self‑service tools — compliance briefly blocks full AWS data migration.
AI-written summary · about
chapters (7)
- 0:00Introduction to the challenge
- 2:20DevOps evolution at Sky
- 4:52Identifying the bottleneck
- 10:12Learning and strategy
- 13:33Product over project mindset
- 19:23Implementation of new workflow
- 23:11Outcomes and future outlook
related talks
26mDevOps in a Data Warehouse: Inside Out
30mDevOps For Data Teams
30mWhat's It Like When DevOps Is Just Another Day at the Office?
30mDevOps is to Infrastructure as Code, as DataOps is to...?
51mFrom DevOps for data to DataOps
30mDevOps to DataOps
44mBreaking a Monolithic Devops
29mDevOps for Data Science? - automate the boring stuff and leverage the OSS ecosystem