You Have A Data Lake, Now What?
summary
A practical roadmap: inventory sources, pick start with the easiest fastest use case (example: helpdesk ticket dashboard), ELT raw into a warehouse, then transform in SQL and version-control the scripts — ELT not ETL — build a living data dictionary, iterate the cycle 10–20 times, and promote a unified star schema. Recommended stack and tools noted: Stitch to Postgres, Amazon Redshift for warehousing and Looker for governed self-serve BI. Outputs are a versioned ELT pipeline, accurate timely reports, and reduced reconciliation pain.
AI-written summary · about
chapters (29)
- 0:00Intro
- 1:19Who am I
- 1:38Reality Check
- 1:52What are the things that make data lakes valuable
- 2:39What makes a data lake valuable
- 4:00Business Intelligence
- 4:27Philosophical
- 4:37Data Dictionary
- 5:21Transforms
- 6:14Schema
- 7:15Why you are learning
- 7:48Definitions
- 9:09Practicality
- 10:11What do I end up with
- 10:51What does it matter
- 11:49Easy fastest use case
- 12:22helpdesk ticket dashboard
- 13:16Dimensions and measures
- 14:16Take your medicine
- 15:25Lowhanging cases
- 15:53Interdisciplinarity
- 16:56BI Tools
- 17:26Maintenance Mode
- 18:24What will come back to bite me
- 21:40Data problems
- 22:42Troubleshooting strategies
- 23:37Automation
- 24:19Collaboration
- 25:06ETL
related talks
25mBuild open format data lakehouse
36mThe Data Lakehouse
02h 05mBig Data and the Data Lake from the Analyst Perspective
11mBreaking Data Silos: Enterprise Data Management
28mData Reporting and Analytics with API's
34mData Engineering Architecture
58mUnderstanding Big Data for Software Engineers
21mUpgrading from the modern data stack to the modern data lake