DataEngBytes 2023 - MEL-T3-06 - David Colls

on YouTube ↗31m

" summary

Maps seven specific Lean wastes onto data production, shows how value‑stream mapping and data as a product stop overproduction and inventory, and demonstrates practical fixes: a streaming rebuild that cuts delivery from three months to 15 minutes, detection of quality issues within that window, and program improvements that reduce failure demand from 60 to 20, double delivery capacity, cut incident detection from hours to minutes and slash manual testing by 80%.

AI-written summary · about

" chapters (17)

1. 0:00Introduction
2. 1:02Data pipelines meaning
3. 6:16Overproduction in pipelines
4. 8:21Inventory waste in data
5. 11:31Overprocessing untrusted data
6. 13:16Transportation and lineage loss
7. 14:41Motion waste in workflows
8. 15:46Waiting costs in data work
9. 17:31Defects cumulative impact
10. 18:57How wastes compound
11. 20:23Five lean principles
12. 22:33Value as a product
13. 23:58Value stream mapping
14. 25:42Flow in data pipelines
15. 27:30Pull in consumption
16. 28:33Continuous improvement
17. 29:37Further reading

" related talks

• 29mDataEngBytes 2023 - SYD-T1-08 - Rishu Saxena & John CosgroveRishu Saxena, John Cosgrove · DataEngBytes Sydney 2023
• 35mApplied Data ScienceGiovanni Lanzani · PyData Amsterdam 2017
• 14mHere, There and Everywhere: Spotting Data Opportunities 101Mor Hananovitz · hayaData 2024
• 06mIgnites - Value Stream Mapping & AnalysisJohn W Spangler · DevOpsDays Rockies 2016
• 27mOptimizing the engineering value streamPrem Chandrasekaran · DevOpsDays Washington, D.C. 2024
• 15mOptimizing the engineering value streamPrem C · XConf North America 2021
• 31mEngineering Effectiveness: Achieving your goals with lessRyan Murray · ParadigmShift 2023
• 24mA Practical Approach To Unlock Value From Data and AnalyticsMaria Feria · PyData Global 2022