Snowpipe Fundamentals: Ingesting Streaming Data into Snowflake Made Easy
summary
Step‑by‑step setup shows a working pipeline: create an S3 bucket, attach an IAM policy and role, create a Snowflake storage integration and external stage, then a pipe that uses SQS/SNS (or Lambda REST calls) to trigger automatic COPY INTO loads. The demo uploads files and they land in Snowflake near‑real‑time (<1 minute); pipes support pause/resume, ownership transfer, monitoring queries and SNS error notifications, and require only per‑second serverless compute.
AI-written summary · about
chapters (14)
- 0:00Introduction
- 17:00Snowpipe fundamentals
- 20:34Snowflake architecture
- 29:01Snowpipe features
- 34:21Snowpipe prerequisites
- 37:19S3 bucket setup
- 45:59Storage integration
- 50:59Table and external stage
- 1:00:12Snowpipe object
- 1:05:03Event notifications
- 1:09:26Monitoring Snowpipe
- 1:17:07Live demo
- 1:20:40Best practices
- 1:23:11Q&A
related talks
01h 20mSupercharge Your Real-Time Analytics: Snowpipe Streaming & Dynamic Tables in Snowflake
13mStreamline workloads with deep integration from Snowflake
28mScalable AI & Web Insights with GitLab, Snowplow & Snowpipe
24mA Single Data Platform for All of Your Workloads
21mNext-Level API Management: Ensuring Data Security & Efficiency
27mDataOps & Full Snowflake Automation In 1 Click
28mBuilding A Governed Data Lake
39mUsing Streaming Data Pipelines in the Cloud to Achieve Success