data Artisans Product Announcement

on YouTube ↗37m

" summary

Announces dA Platform 2 that packages Apache Flink with an Application Manager that treats jobs as declarative applications, tracks metadata and savepoints and forks, and automates suspend/restore/upgrade workflows (stateful suspend, cancel-without-state, periodic savepoints). Integrates with Kubernetes, ELK and Influx by default, and hooks into GitLab/Jenkins for CI-driven deployments. Architecture favors one job manager per job, stores metadata (Cassandra), is cloud-ready, closed-source App Manager, and offers early access with GA planned by end of year / early next year.

AI-written summary · about

" chapters (5)

1. 0:00Introduction and overview
2. 1:05Lessons from streaming architectures
3. 7:31Introducing DA platform
4. 11:25Application Manager demo
5. 28:45Closing and Q&A

" related talks

• 30mData Platforms Made Easy - Introducing TRAC D.A.P.Martin Traverse, Greg Wiltshire · Open Source in Finance Forum London 2022
• 34mTransforming Data Processing with Kubernetes: Journey Towards a Self-Serve Data MeshRakesh Subramanian Suresh, Jainik Vora · Data on Kubernetes Day North America 2023
• 37mThe Future of Apache FlinkAljoscha Krettek · Flink Forward Berlin 2016
• 52mBetter ETL with Managed Airflow in ADFNiall Langley · SQLBits 2024
• 42mApache Flink deep-dive #bbuzzStephan Ewen · Berlin Buzzwords 2015
• 40mFrom concept to production; the fastest way to develop & deploy Apache Flink jobsSteffen Hausmann, Danny Cranmer · Flink Forward Global 2021
• 20mData-as-a-product: A framework for collaborative data wranglingClara Amorosi · posit::conf(2025)
• 08mInterview with Jamie GrierJamie Grier · Flink Forward Berlin 2016