Serverless SQL Pipelines
summary
Concrete demos and numbers: a recorded BigQuery run processes several terabytes (tens of millions of rows aggregating to ~2,500) in ~30 seconds for roughly $25, and BigQuery ML shows CREATE MODEL (logistic regression) with AUC ≈ 0.9. Covers Amazon alternatives with S3 Select quick CSV peeks, Athena full ANSI SQL over S3, Glue as catalog+Spark ETL, plus Aurora Serverless and Redshift Spectrum. Focus is vendor comparisons, pipeline patterns, costs and when each tool fits.
AI-written summary · about
chapters (7)
- 0:00Start
- 16:13Athena Details
- 16:53Cost Example - Athena
- 17:08RDS Aurora Serverless
- 29:15AWS Pipeline
- 33:07GCP Pipeline
- 43:33Serverless SQL Services
related talks
25mA simple serverless data pipeline
25mData Gone in 60 Seconds: A Serverless ETL Heist
38mUnleashing the serverless data pipelines
34mDesigning Data Intensive Applications In Serverless Architecture
28mServerless Data Analysis and Reporting using SQL
34mBuilding scalable end-to-end deep learning pipelines in the cloud
42mBuild and automate a modern serverless data lake on
01h 00mServerless - reality or BS - notes from the trenches