Data Teams Summit 2023
as the organisers put it
Data Team Summit is an annual virtual community event aimed at helping DataOps professionals build, manage, and monitor data pipelines.
| title | author | length |
|---|---|---|
| Infrastructure scale issues | Mark Freeman II, Richad Nieves-Becker, Ben Doremus, Sarah Floris | 30m |
Early signs: pipelines silently fail, dashboards go stale, queries slow, backlog velocity collapses and a full-time hire is loading the initial data. Concrete… | ||
| Engineering practices at Maersk | Mark Sear, Kunal Agarwal | 29m |
Maersk turns data into operational levers: a single E‑class ship moves 18,000 containers and one journey can exceed 19th‑century transcontinental volumes, so… | ||
| DevOps to DataOps | Ali Khalid | 30m |
Argues that DataOps equals DevOps + data observability, with practical steps: modular, loosely coupled pipelines and data products; put all configs in Git and… | ||
| DataOps for digital twins | Pragyansmita Nayak | 30m |
Digital twins depend on real-time trusted data and close-loop feedback from sensors plus enrichment sources; the argument is that scalable deployments require… | ||
| The mesh inside the mesh | Christian Barca | 31m |
Migrates a monolithic ETL into a domain-oriented decentralised design that processes research data harvested from over 2,000 repositories, producing tens of… | ||
| Asset based orchestration: the future of data orchestration | Jonathan Neo | 24m |
Claims that shifting from task/DAG scheduling to asset-based orchestration fixes dbt scale problems by importing assets across repos, building a global… | ||
| Data contracts: driving data culture change | Andrew Jones | 26m |
GoCardless implements data contracts since 2021 to stop surprise schema changes and restore trust: it plumbs CDC into BigQuery to recreate source tables, then… | ||
| Data quality as a product | Alejandra Cabrera, MONA RAKIBE | 21m |
Clearbit treats data quality as a product: using Telmai’s observability platform and a TM subscription to monitor 50 million domains, it builds a Data Quality… | ||
| Time series data science problems with InfluxDB | Anais Dotis-Georgiou | 28m |
Shows how InfluxDB uses Telegraf exec processor to run Python at the edge and produce micro‑batch forecasts for a home‑brew temperature control use case, using… | ||
| Good habits for data engineers | Loris Marini, Swati Vishwanathan, Tobias Zwingmann, Vishal Ramrakhyani | 29m |
Panel lists concrete habits: build network before you need it by doing low‑stakes non‑transactional outreach and internal events; use a recurring Friday quiz… | ||
| Evolution of analytics at Wattpad | Preeti Hemant | 28m |
Wattpad runs on ~90M users and finds analytics time split 70% ad hoc, 20% experiments, 10% deep analytics, which makes analytics a service desk. The team… | ||
| Data engineering team leads | Matthew Edwin Weingarten | 23m |
Career advice for new data engineering leads that focuses on concrete practices: prioritize people through regular one-on-ones and delegation, run monthly… | ||
| Data mesh unlocking growth | Nikolas Schriefer | 27m |
Linkfire reorganizes around domain value streams, moves ownership of ~50 data assets to cross‑functional teams, and enforces federated governance via a… | ||
| FinOps for business results | Thiago Bittencourt Gil, Clinton Ford | 19m |
Claims concrete savings and steps: data management ~40% of the cloud bill, organizations run on average 133% over budget, and one EMR environment sees EMR… | ||
| Building flexible data teams to improve product delivery | Julia King, Elliot Shmukler, Stephan Claus | 30m |
Panel lays out when to centralize versus embed: choose centralized unless embedded domains are large enough to support career growth, or when you can form a… | ||
| Better data quality with ML & visualization | Zach Wright | 30m |
Anomalo monitors an enterprise warehouse with unsupervised machine learning that automatically flags unknown anomalies, routes alerts to Slack/PagerDuty/Teams… | ||
| Future of data engineering: what's in store? | Eric Callahan, Brandon Beidel, Shane Murray | 31m |
Panel predicts tight budgets force teams to do more with less, driving investment in orchestration and unification tools to replace manual stitching. Specific… | ||
| Self service data governance | Pravin Kedia | 29m |
Uses Watson Knowledge Catalog on Cloud Pak for Data with Snowflake and SQL virtual tables to enforce data quality: normalizes vendor names (Corp → Corp… | ||
| IC to manager panel discussion: Bridging the gap going | Matthew Blasa, Monica Kay Royal, Puppy Tsai, Rob Albritton | 28m |
Transitioning from individual contributor to manager means trading deep technical hours for people and process work: hiring risk becomes personal, hiring… | ||
| Situational awareness in a technology ecosystem | Charles Boicey | 28m |
Claims a healthcare data stack that achieves near-continuous uptime by combining instrumentation, observability and human-in-the-loop intelligent assist atop a… | ||
| Building the best data team is not rocket science | Nirmal Budhathoki | 30m |
Argues that the right data team aligns responsibilities to hires and gives concrete hiring rules: start centralized/decentralized/federated based on company… | ||
| Data team strategies: unleashing your team | Sanjeev Mohan, Benjamin Rogojan, Kunal Agarwal | 27m |
Claims concrete steps: many organizations run 100+ pipelines and even small firms hit that scale, and a survey finds ~30% use custom checks, ~30% buy tools… | ||
| nothing matches | ||