~/talk

Snowflake and dbt -- Our Journey to the Cloud

on YouTube 21m
Talkat DataEngBytes 2021 · on Sunday, 2021-10-10· [data-engineering]

speakers

name:Philipa Sidaway
links:none on file
name:Juliana Zhu
links:none on file

resources

video:https://youtu.be/vqS7aMmQQlE
slides:none on file

summary

Migration uses Snowflake plus dbt with Git and buildkite CI/CD, Snowpipe from S3 (kinesis notifications) and 5 tran connectors. Pipeline changes reduced 8 hours to 30 minutes for a monthly job, uses Snowflake zero copy cloning for cheap dev datasets, and appends branch names to schemas for isolated builds. Quality is enforced with dbt tests and sqlfluff linting, docs are aggregated in a central dbt mechadocs repo.

AI-written summary · about

chapters (7)

  1. 0:00Introduction
  2. 1:14Data as a Product
  3. 2:43Snowflake
  4. 5:38Snowflake Roles
  5. 7:40DBT
  6. 12:14Continuous Integration
  7. 15:15Data Testing

related talks

~/search
type to search 6,249 conferences, 138,987 speakers and 185,172 talks
~/help
cmdctrl k
search every conference, speaker and series
/
focus the filter search
h
home
c
conferences
s
speakers
e
series
t
light / dark — the status bar button does the same
?
this help
esc
close this, then clear every filter — or just the search, while you are typing in it

Ignored while you are typing. Pages without a filter rail have no search to focus.