What is a Data mesh?
summary
Claims that data mesh is a socio-technical pattern borrowed from domain driven design and meant to fix organizational coupling by giving each data domain domain ownership as a product, backed by a self-serve platform and federated computational governance. Shows a concrete stack: catalog + orchestrator + cache + Spark on EMR feeding S3/Redshift/Aurora/Postgres, running in containers, and demos Google product GCP Dataplex as a ready example of the logical layer.
AI-written summary · about
chapters (14)
- 0:00Data mesh concept
- 1:43Core problem data mesh solves
- 6:26Limitations of ETL pipelines
- 10:21Organizational data challenges
- 12:30Domain-driven design principles
- 20:06DDD for analytics
- 21:52Socio-technical solution
- 22:34Four pillars of data mesh
- 26:33Building a data mesh catalog
- 29:47Orchestrator role
- 32:15Single domain architecture
- 37:43Google Cloud Dataplex example
- 47:49Data quality and governance
- 50:45Conclusion & Q&A
related talks
46mA Data Mesh Reference Implementation on AWS
39mData Mesh in the wild
54mData Mesh in Practice
42mFour Years of Data Mesh in Practice
30mFrom Data Lake Entanglement to Data Mesh Decoupling: Scaling a Self-Service Data Platform
42mFour Years of Data Mesh in Practice
40mData Mesh, Data Fabric and DataOps; what should you choose?
47mData Mesh in Kubernetes