Leaving ETLs Behind: Moving from a ‘Code Less’ Solution to a ‘Code Full’ Approach
summary
Migration from an IBM DataStage GUI-based ETL to a code-first ELT stack using dbt and Snowflake, arguing that "code less" ETL tools struggle with modern development practices like Git workflows and incremental delivery. Covers why they migrated, compares ETL vs ELT paradigms, shows benefits of a "code full" approach for developers and analysts, and describes improvements in production run monitoring, alerting, incident resolution, and making data accessible rather than delivering datasets; includes a post-migration experience report and observed gains.
AI-written summary · about
chapters (10)
- 0:00Introduction
- 1:52ETL classic problems
- 8:17Modern ELT approach
- 11:30DBT as transformation
- 19:36Snowflake as target
- 22:24DBT + Snowflake architecture
- 27:09Code full migration benefits
- 30:18Progressive migration challenges
- 33:52DBT philosophy shift
- 34:14Q&A
related talks
01h 28mA Guide to the Modern Data Stack
16mETL vs ELT: Why the T has shifted
38mReigniting the ETL vs. ELT Debate - Which Better Supports the Business?
41mModern Data Architectures – What They Are and How to Implement?
30mA Devops Approach to Data: A Tale Through the Trenches in Search of a Cloud Data Revolution
38mThe Rise & Downfall of the Data Engineer REVISITED
39mWhat if SQL was back in fashion?
55mThe ELT pattern in action with DBT and SQLMesh