Crafting modern data platforms: reveal your data value
summary
Provides a concrete, repeatable platform pattern: ingest with Kinesis Data Firehose or AWS Glue jobs, land into a data Landing Zone with Glue crawlers and DataBrew sampling, run quality checks with Glue Data Quality, expose integrated tables via Athena/Redshift and self‑service via QuickSight Q, and serve low‑latency products from DynamoDB or OpenSearch behind API Gateway and Lambda. Implements lineage using OpenLineage + Airflow and a Marquez UI on Fargate; result: a federated raw → integrated → consumable layout that surfaces which customers are affected and enables product fixes that reduce churn.
AI-written summary · about
chapters (8)
- 0:00The pillars of data strategy
- 3:10Defining a data mindset
- 6:47Identifying key personas
- 12:06Removing adoption barriers
- 20:27Enabling flexible consumption
- 30:37Managing data consistency
- 34:54Scaling with distributed models
- 37:23Summary and closing
related talks
59mModern data lakes with Lake Formation, Amazon Redshift & AWS Glue
29mGrowing insights through hybrid data architectures
27mRecipe for Building a Discoverable and Governed Data Platform.
56mUnderstanding and achieving a modern data architecture
28mTurn Data Into Insights
10mBuilding a Data Platform at Assembly Payments
16mAutomated Data Analytics
43mManaging a Data Lake on AWS