Data team strategies: unleashing your team
summary
Claims concrete steps: many organizations run 100+ pipelines and even small firms hit that scale, and a survey finds ~30% use custom checks, ~30% buy tools, ~30% do nothing. Teams deploy data observability and FinOps to catch runaway queries (examples: recurring $100/day jobs or $100k weekend bills), move toward serverless and data sharing to cut SFTP churn, and prioritize a multicloud control plane, unified metadata stores, and productized datasets with versioning and SLAs.
AI-written summary · about
chapters (7)
- 0:00Introduction to speakers
- 1:28Ensuring pipeline reliability
- 6:20Managing cloud costs
- 13:03Talent shortage challenges
- 18:22Role of automation in data ops
- 21:56Predictions for 2023
- 26:52Future of data teams
related talks
38mKeeping pace with data infrastructure so it doesn't outrun you
23mData Team as an Optimisation Problem
42mBringing Software Engineering Rigour to Data, DataEngBytes 2022
39mBREAKING YOUR DATA TEAM OUT OF THE SERVICE TRAP
11mTransforming Problem-Solving with Data-Driven Insights
07mFrom Scaling APIs to Scaling Data to Scaling Your Team
37mTaming the Data Tsunami
16mData pipeline automation with built-in metadata and lineage (PRT054)