Open Source Powers The Modern Data Stack

on YouTube ↗29m

" summary

Argues that open source components let teams build a tailored modern data stack and gives concrete numbers and tools: Airbyte with 200+ connectors and 9,500 Slack members, Airflow orchestration, dbt transforms and Superset for visualization, with plans for reverse ETL in 2023. Demo shows a Salesforce → Snowflake pipeline, selectable staging, replication modes including incremental CDC and full refresh, visibility into sync logs, and a connector development kit with unit tests and sandboxing for custom connectors.

AI-written summary · about

" chapters (6)

1. 0:00Introduction
2. 1:47ETL vs ELT explained
3. 5:40Modern data stack overview
4. 8:09Key open source components
5. 16:17Live demo of Airbyte
6. 25:09Q&A session

" related talks

• 32mBig Data With Open Source SolutionsIgor Khrol · Big Data Conference Europe 2023
• 34mQuantifying the growth and evolution of open-source communities with Apache SupersetRobert Stolz · ApacheCon@Home 2021
• 15mOpen Source in a Data-Driven WorldXu Chen · LinuxCon + ContainerCon + CloudOpen China 2017
• 44mOpen Source in the Age of Cloud and MicroservicesMichael Meskes · Open Source Summit Europe 2023
• 44mPanel Discussion: How Open Source Helps DataOpsMandy Chessell, David Radley, Cheranellore Vasudevan, Dan Wolfson · Open AI + Data Forum @ Open Source Summit North America 2022
• 29mNavigating the Modern Data Stack with Open Source Headless BIArtyom Keydunov · OSA Con 2022
• 25mState of Open Source DatabasesPeter Zaitsev · OSA Con 2022
• 50mMasterclass: Building a data stack in 2021Ryan Janssen, Paul Blankley · Big Data & AI Toronto 2021