Platform Oriented Data Architecture for Acceler...
summary
Argues that platforms must own both operational and analytical data and publish high-quality data product interfaces, not just APIs, to eliminate the “mickey mouse” quality friction of monolithic data lakes. Describes using Kafka for streaming, cloud object storage and BigQuery/MySQL for stores, and automating deployment with Terraform and Airflow plus Kubernetes for repeatable pipelines. Central data org provides governance and a self-service data infrastructure so domain teams fix data where it’s produced.
AI-written summary · about
chapters (6)
- 0:00Start
- 3:53The Microservice Architecture
- 8:28Data Lake
- 13:29Data Interfaces
- 22:09The Data Dictionary and the Data Catalog
- 25:14Future of Data Engineering
related talks
36mIntroducing Data Mesh A Paradigm Shift In Analytical Data Management Architecture
47mModern Data Architecture: Breaking the Big Data Monolith
35mCloud Native Data Platforms - The why, what and how
38mPlatform Building for Data Mesh - Show me how it is done!
47mDataOps for AI-Ready Platforms
01h 00mData Mesh: Beyond the Data Lake
48mData Mesh Paradigm Shift in Data Platform Architecture -
39mCrafting modern data platforms: reveal your data value