Managing Apache Flink on Kubernetes - FlinkK8sOperator
Talkat KubeCon + CloudNativeCon North America 2019 · on Tuesday, 2019-11-19· at 15:20· Room 6D - San Diego Convention Center, Upper Level·
summary
An open-source operator uses a FlinkApplication CRD to create one Flink cluster per CR and manages the full lifecycle via a controller runtime state machine (new → create cluster → submit → running → updating). It requires a runnable image that can start jobmanager and taskmanager, records health and metrics in the CR status, and performs zero-downtime update by launching a second cluster, taking a savepoint in S3, switching jobs, and auto‑rolling back on failure, yielding faster scaling, easier config, and measurable cost savings.
AI-written summary · about
chapters (25)
- 0:00Introduction
- 0:22Agenda
- 2:19Why Flink
- 4:36How Flink is run
- 5:31Flink Architecture
- 6:33Flink Deployment
- 8:33Flink Deployment Problems
- 11:14Flink K8sOperator
- 13:15Architecture
- 15:00Control Loop
- 16:15State Machine
- 19:24Demo
- 25:30Impact
- 28:01Current State
- 28:55Flink Talks
- 29:25Happy Hour
- 30:00Flight Stickers
- 31:48Save Point
- 32:35Restore Path
- 33:20Operator SDK
- 34:18Clusters
- 34:36User Interface
- 35:29Overhead
- 37:45Autoscaling
- 40:52Infraspinatus
related talks
22mBuilding a Kubernetes Operator for Apache Flink in Java
41mRunning Apache Flink everywhere
16mZero-downtime upgrades of Flink applications
26mRunning Apache Flink on AWS
26mKubernetes & real-time analytics: How to connect these worlds with Apache Flink
37mIntegrate Flink with Kubernetes natively
33mFlink in Containerland
20mAutoscaling Apache Flink Applications