KEYNOTE: How We Developing Data Services/Products in GitLab.com
summary
GitLab.com describes an almost-100%-open-source enterprise data platform that dogfoods its own product, handles 13+ million users with a 1,600+ fully remote team, and releases every 22nd. The stack foregrounds dbt for SQL transformations, Meltano as pipeline-as-a-service, Airflow orchestration, Snowflake storage with single-copy clone test environments, and Monte Carlo for ML-driven data observability. Pipelines auto-trigger by changed artifacts, strict CI gates enforce quality, and data-science projects like propensity-to-expand and churn prevention already run in production.
AI-written summary · about
chapters (18)
- 0:00Keynote introduction
- 2:39About GitLab and speaker
- 4:03GitLab mission and products
- 9:20Data platform mission and challenges
- 13:16Open source culture and dogfooding
- 14:22DevOps culture and workflow
- 17:30GitLab data stack and tooling
- 19:57DBT and its benefits
- 20:39Montana pipeline as a service
- 22:26Monte Carlo for observability
- 24:12Automated pipelines and testing
- 26:58Data team roles
- 27:19Data science projects
- 30:10ELTD documentation and strategy
- 32:42Key takeaways and future
- 38:27GitLab handbook importance
- 40:36Closing remarks
- 41:39Q&A
related talks
25mHow we create and leverage Data Services in GitLab.com
27mDo the magic with DevOps, Open Source and All-remote teams (with Python, of course)
31mHow to Implement DataOps using
22mProduct Keynote
31mAI-driven DataOps: Building data products with a taste of the future
14mKeynote Analysis
21mDriving Cultural Change to Improve Software Development and CI/CD
46mFrom Dev to Prod with GitLab CI