Data Mesh in Kubernetes
summary
PwC Germany builds an internal Data Ecosystem on Kubernetes running on Azure with about 10 engineers, claiming domain teams own data as products and self-serve via Backstage. Concrete stack includes Ranger Keycloak DataHub Atlas Great Expectations for governance and Trino Superset Hive for query, with Kafka Airflow Argo DBT and Camel K for pipelines. Real problems are operator incompatibilities during Kubernetes upgrades, many tiny databases, vendor lock‑in and heavy maintenance; some tools get retired but the platform mostly delivers discoverability and governed reuse.
AI-written summary · about
chapters (7)
- 0:00Introduction
- 1:04Agenda and motivation
- 8:16What is data mesh?
- 15:41Kubernetes components
- 23:20Implementation challenges
- 35:37Tool selection
- 40:20Q&A
related talks
34mTransforming Data Processing with Kubernetes: Journey Towards a Self-Serve Data Mesh
54mData Mesh in Practice
40mData Mesh, Data Fabric and DataOps; what should you choose?
42mFour Years of Data Mesh in Practice
12mMDM: K11s Orchestration for Enterprise Data Quality at Scale
05mProject Lightning Talk: Kubernetes Ontologies With Meshery
07mLightning Talk: My Database Runs on Kubernetes. What's Next? Data Platforms!
28mProgress For Big Data In Kubernetes