Notebook to Pipeline: Hands-On Data Engineering w Python
summary
Claims a complete notebook-to-pipeline run with live code and reusable templates, using Python, Snowflake and Snowpark, plus a 120‑day Snowflake trial for cloud compute. It ingests CSVs from S3 with the copy into command, loads about ~1 billion rows across ~400 files, transforms with declarative dynamic tables that refresh incrementally, builds a semantic model via Cortex Analyst, and wires a Snowflake Intelligence agent that answers queries and renders a chart, demonstrating the pipeline actually works.
AI-written summary · about
chapters (7)
- 0:00Introduction
- 14:31Setup and environment
- 27:51Snowflake connection and objects
- 33:50Data ingest with COPY INTO
- 57:56Dynamic tables for pipeline
- 1:03:42Semantic model and AI workflow
- 1:08:14Q&A
related talks
31mHow to quickly build Data Pipelines for Data Scientists
35mData Processing Pipelines for Small Big Data
01h 28mMaintainable code in data science
42mBuilding a Data Driven Organization
24mSmart Cities Unleashed
45mWorkshop 3: Data Engineering - A Roadmap... by Ramon Perez [Part 3]
39mData and ETL with Notebooks in Papermill
53mEnterprise-Ready Python for High-Performance Data Teams