Why Most Data Projects Fail and How to Avoid It
summary
Eighty-five percent of data projects fail and only 15% make production, backed by Gartner and a survey of 81 respondents, so the concrete fixes matter: use the right team ratios (5 data engineers per data scientist), pick tools for the use case not the vendor (Snowflake Databricks Spark data lake examples), aim for a demonstrable timeline and a 10x ROI target, and deliver runnable production pipelines with ops in place rather than prototypes.
AI-written summary · about
chapters (14)
- 0:00Introduction
- 1:50Technology is only part
- 2:54Critical roles of data teams
- 6:49Defining business value
- 8:44Realistic timelines for value
- 11:32Choosing execution environment
- 13:18Focusing on singular goals
- 14:23True value of data
- 17:10Five key questions
- 19:18Inverting problems for solutions
- 22:50Avoiding copied architectures
- 24:40High versus low value teams
- 29:40Unified role of data science
- 31:05Q&A
related talks
32mWhy Most Data Projects Fail and How to Avoid It
53mThe State of Data Engineering
25mWhy Software Engineering Best Practices Fail in Data Engineering
48mBecoming a Data Head, a reflection on the trade offs and decision
41mBreaking the 80/20 Dilemma in Data Science Projects
49mKeep it Clean: Why Bad Data Ruins Projects and How to Fix it
30mDevOps For Data Teams
30mFailing Fast with Data: Bringing CI to the Land of Data