DevOps to DataOps
summary
Argues that DataOps equals DevOps + data observability, with practical steps: modular, loosely coupled pipelines and data products; put all configs in Git and treat infra as code; implement unit/job/integration tests so CI/CD becomes single-touch deployments that let teams move from monthly to daily releases. Emphasizes prioritizing data quality, creating data contracts with sources and consumers, assigning severity-tagged alerts, and sharing quality results with business so problems surface and get fixed quickly.
AI-written summary · about
chapters (19)
- 0:00Introduction to DataOps
- 1:07Why DataOps differs from DevOps
- 3:39DevOps plus data observability
- 5:07DevOps culture for analytics teams
- 6:11Modularized architecture in pipelines
- 8:19Branching and merging strategies
- 9:23Infrastructure as code
- 10:06Automated testing in analytics
- 11:32Robust CI/CD pipelines
- 13:44Smaller and quicker releases
- 14:49Value of early production deployments
- 15:52Data observability principles
- 16:56Scaling data quality
- 18:22Investing in data quality
- 20:30Source system data issues
- 22:18Data contracts and validation
- 24:50Alert severity and escalation
- 26:39Sharing results with business users
- 28:45Key takeaways and closing
related talks
51mFrom DevOps for data to DataOps
37mWhat is DataOps? Becoming a Truly Data-Driven Organization
30mDataOps: Much More Than DevOps for Data
30mDevOps For Data Teams
30mDevOps is to Infrastructure as Code, as DataOps is to...?
15mDevOps: Beyond the scope of development and operations
59mDevOps for Data Science
20mDevDataOps - Using data for exponential growth of your organization