Introducing Flink on Mesos

on YouTube ↗41m

" summary

Flink gains native Mesos deployment in the Flink 1.2 line with a working demo that launches a Mesos session producing a jobmanager and taskmanager pair and successfully runs a WordCount job. Resource packing uses the Netflix FENsO optimizer to solve the knapsack placement problem across fragmented offers. A long‑running dispatcher for multi‑cluster management exposes a REST gateway so CLI can spawn persistent clusters; Docker and local disk reservation are planned, and dynamic within‑job scaling is in progress.

AI-written summary · about

" chapters (10)

1. 0:00Introduction to Flink on Mesos
2. 3:56Why run Flink on Mesos
3. 8:51First Flink job on Mesos demo
4. 10:16How Mesos integration works
5. 15:40Optimizing task placement
6. 22:56Role of the Flink dispatcher
7. 29:43Elastic Flink clusters on demand
8. 32:52Future of Flink on Mesos
9. 37:09Comparison with Kubernetes
10. 40:47Q&A and closing

" related talks

• 43mApache Flink Meets Apache Mesos and DC/OSJörg Schad · MesosCon Europe 2017
• 30mDemystifying Deployments: Applications or Clusters, Active and Reactive Scaling - What is it about?David Moravek · Flink Forward Global 2021
• 46mFlink SQL in 2020: Time to show off!Timo Walther, Fabian Hueske · Flink Forward Virtual 2020
• 40mFrom concept to production; the fastest way to develop & deploy Apache Flink jobsSteffen Hausmann, Danny Cranmer · Flink Forward Global 2021
• 42mApache Flink deep-dive #bbuzzStephan Ewen · Berlin Buzzwords 2015
• 22mDo Flink on Web with FLOWDongwon Kim, Haemee Park · Flink Forward Europe 2019
• 36mApache Flink in the Cloud-native EraStephan Ewen · Flink Forward Global 2021
• 42mManaging Apache Flink on Kubernetes - FlinkK8sOperatorAnand Swaminathan · KubeCon + CloudNativeCon North America 2019