Data Modelling is not a one man job!

on YouTube ↗22m

" summary

Argues that data modelling must be collaborative and automated, showing how WhereScape 3D plus GIT pipelines let multiple designers share models, enforce code standards, run data tests and deploy to a physical platform. Gives concrete failure example where a bank’s load used 3x data volume and crashed nightly loads, cites missing business rules and tooling limits as common causes, and says automated CI/CD from design to deploy prevents those missteps.

AI-written summary · about

" chapters (11)

1. 0:00Data modeling introduction
2. 3:40One-person modeling challenges
3. 6:07Collaborative requirements gathering
4. 8:35Involving data engineers
5. 10:22Modeling pitfalls in SDLC
6. 12:50Team dynamics and skills gaps
7. 14:56Tooling limitations
8. 17:47Effective modeling tools
9. 19:13Departmental alignment
10. 20:14Automated deployment
11. 20:56Q&A

" related talks

• 39mPanel DiscussionJoe Reis, Zainab Maleki, Jonathan Neo, Ash Smith, Peter Hanssens · DataEngBytes Perth 2023
• 30mDATA MODELLING IS DEAD! LONG LIVE DATA MODELLING!Joe Reis · DataEngBytes Sydney 2023
• 26mDataEngBytes 2023 - MEL-T1-01 - Joe ReisJoe Reis · DataEngBytes Melbourne 2023
• 24mMake data central featureAntoni Ivanov · DSC Europe 23
• 26mData, It’s a People Problem: Understanding the Sociotechnical Elements of a Data-Driven OrganizationGunu Diamond · DataEngBytes Melbourne 2024
• 27mDataEngBytes 2023 - BRI-T1-07 - Joe ReisJoe Reis · DataEngBytes Brisbane 2023
• 43mData Modelling for Startups: Domain Models & the Modern Data StackThomas in't Veld · Crunch Conference 2022
• 31mExecuting An Effective Data Strategy: Why Automation Is No Longer A ChoiceRamesh Panakkal · Big Data LDN 2022