Do the Magic with All-Remote Data Teams...

on YouTube ↗40m

" summary

All-remote GitLab Data runs on GitLab itself and balances cost with speed: 600+ people across 65+ countries, 30M users, monthly releases on the first Thursday. Platform components: Airflow for orchestration, Snowflake as single source of truth, Snowplow for event capture, Spark for heavy processing, Terraform + Kubernetes for infra, and a GitLab-built declarative RBAC tool for Snowflake. Outcomes: automated pipelines, dbt for collaborative SQL models, Monte Carlo to auto-detect data anomalies, zero-copy clones for fast sandboxes, and production ML like propensity and churn scores that materially inform product decisions.

AI-written summary · about

" chapters (13)

1. 0:00Introduction
2. 1:03Data team mission and values
3. 3:53All-remote company scale
4. 7:42Data team mission and pyramid
5. 11:15DevSecOps and GitLab dogfooding
6. 14:29Data platform architecture and tools
7. 20:20DBT and Meltano in data
8. 22:26Data observability and roles
9. 26:38Data science projects and structure
10. 29:07Documentation and process discipline
11. 30:32Data team strategic direction
12. 34:23Recap and contribute to open-source
13. 35:07Q&A

" related talks

• 28mScalable AI & Web Insights with GitLab, Snowplow & SnowpipeVed Prakash · DSC Europe 24
• 25mHow we create and leverage Data Services in GitLab.comRadovan Bacovic · DSC Croatia 2022
• 31mHow to Implement DataOps usingEmilie Schario · GitLab Commit San Francisco 2020
• 28mUsing Dataops To Build And Manage Data Products On SnowflakeOmar Khawaja, Guy Adams, Paul Rankin · Big Data LDN 2022
• 17mStrengthening Security and Compliance in Our Data PlatformDennis Van Rooijen · DSC Europe 24
• 31mAI-driven DataOps: Building data products with a taste of the futureRadovan Bacovic · DSC DACH 25
• 27mDataOps & Full Snowflake Automation In 1 ClickTony Hajdari · Big Data LDN 2022
• 27mHow is it to lead an all-remote, globally distributed team?Dennis Van Rooijen · DSC Europe 23