Scalable Python and SQL Data Engineering without Migraines
summary
Live demo builds a Snowflake pipeline for a franchise food‑truck case: create a database and isolated virtual warehouse, stage S3 files, load ~20 GB in the notebook, use Snowpark to mix Python dataframes and SQL, register a UDF to convert Fahrenheit/inches to Celsius/mm, attach Frostbite weather data from the marketplace without copying, and use streams for CDC plus tasks to ingest deltas into a daily orders per city table. Warehouses scale in ~1s; the demo runs end‑to‑end. Snowpark notebook
AI-written summary · about
chapters (11)
- 0:00Introduction
- 4:05Snowflake core features
- 8:45Data engineering workflow
- 10:31Demo setup
- 13:22Loading data with Python
- 15:51Scaling compute resources
- 17:17External data sources
- 20:46Continuous data loading
- 22:55Finalized data model
- 26:51Analyzing data
- 29:04Q&A
related talks
33mData Engineering simplified with Snowpark for Python on Snowflake
29mScaling Python-Powered Machine Learning With Snowflake
23mAcceleratinag Data Science and Machine Learning with Snowflake DataTech
27mMachine Learning in the Warehouse with Python
29mBuilding End to End Gen AI Powered Analytics: From Data to Production
01h 18mFrom raw data to interactive data app in an hour
01h 20mSupercharge Your Real-Time Analytics: Snowpipe Streaming & Dynamic Tables in Snowflake
21mMachine Learning in your Data Warehouse using Python