Data Mesh & Data Vault

on YouTube ↗31m

" summary

Five-year case study of a cloud migration that centers Snowflake and combines Data Vault 2.0 with domain-driven teams, templates and automation. Ingest uses Fivetran and Confluent Kafka, lineage via a data catalog, transformations are templatized in dbt + automateDV to produce repeatable hubs, links and satellites. Teams train on a shared-kernel governance, run streams in Snowflake, and consolidate multiple legacy sources (Redshift, Aurora) into a single Snowflake account; outcome: faster, scalable onboarding and repeatable delivery of data products.

AI-written summary · about

" chapters (11)

1. 0:00Data Mesh & Data Vault intro
2. 1:13Core principles
3. 2:39Customer journey & cloud
4. 7:37Data Vault basics
5. 9:03Business architecture & domains
6. 12:59Curated zone & raw vault
7. 15:05Data Vault path to glory
8. 17:16Team topologies
9. 20:55Shared kernel & integration
10. 26:33Why Data Vault
11. 28:39Q&A

" related talks

• 17mDataEngBytes 2023 - SYD-T1-05 - Patrick CubaPatrick Cuba · DataEngBytes Sydney 2023
• 01h 03mDiscovering the value of a cloud data platformMichael Oppenheim, Arunashish Majumdar, Chad Duncan · AWS re:Invent 2019
• 27mDataOps & Full Snowflake Automation In 1 ClickTony Hajdari · Big Data LDN 2022
• 21mSnowflake and dbt -- Our Journey to the CloudPhilipa Sidaway, Juliana Zhu · DataEngBytes 2021
• 30mBuilding Data AI Application with the Snowflake AI Data CloudRoy Pasternak · hayaData 2024
• 24mA Single Data Platform for All of Your WorkloadsStephen Weingartner · DataEngBytes 2021
• 01h 17mData Engineering in the AI Era: Building a Robust Foundation for AI ApplicationsArtemiy Kozyr · DSC Europe 24
• 34mTales of Data Architecture EvolutionJosef Goldstein · Build Stuff 2022