DataOps & Full Snowflake Automation In 1 Click
summary
Demonstrates a platform that claims one-click DataOps by auto-creating Snowflake environments (ingest, transform, Data Catalog integration) from a git repo using YAML declarative pipelines and CI/CD; it uses Snowflake zero-copy clone for per-branch databases, orchestrates existing ETL tools (Fivetran/Matillion/Informatica/Talend or custom Python), runs schema and column tests at each step, and stores metadata for governance. Customer example: a pharma move from ~3 months per product to delivering 120 data products per month, showing the approach works.
AI-written summary · about
chapters (15)
- 0:00Introduction to DataOps
- 1:02Challenges in data automation
- 3:28Why DataOps matters
- 5:18Declarative vs imperative
- 8:06Snowflake declarative approach
- 10:53Branching with zero-copy clones
- 13:19Agility with full data sets
- 14:48Git as source of truth
- 16:14Orchestrating ETL tools
- 18:00Git-driven automation platform
- 20:47Pipeline runners and tasks
- 22:12Roche Pharmaceuticals case study
- 23:17Platform demo overview
- 25:23Key benefits of automation
- 26:55Q&A
related talks
31mHow to Implement DataOps using
24mA Single Data Platform for All of Your Workloads
21mSnowflake and dbt -- Our Journey to the Cloud
31mAI-driven DataOps: Building data products with a taste of the future
31mData Mesh & Data Vault
23mBuilding A Code Generated Data Platform
28mScalable AI & Web Insights with GitLab, Snowplow & Snowpipe
27mRecipe for Building a Discoverable and Governed Data Platform.