Tiny Flink — Minimizing the memory footprint of Apache Flink

on YouTube ↗20m

" summary

Explains how to run Apache Flink inside one JVM using Flink’s MiniCluster, trimming JVM native memory by lowering thread stacks, switching GC, and tuning Flink buffers to hit a tiny footprint. Demonstrates launching an empty cluster at about 20 MB heap and a real pipeline that stabilizes with ~100 MB heap and ~250 MB RSS, achieving roughly 100 MB/s throughput on a laptop; provides code, benchmarks, and precautions about stack sizes and GC tradeoffs.

AI-written summary · about

" chapters (4)

1. 0:00Start
2. 5:32Deployment Options for Apache Flink
3. 10:19Benchmarking the Throughput: Setup
4. 13:15Heap vs Native Nemory

" related talks

• 23mDemystifying Flink Memory Allocation and tuningRoshan Naik · Flink Forward Europe 2019
• 42mApache Flink deep-dive #bbuzzStephan Ewen · Berlin Buzzwords 2015
• 43mTo Petascale and Beyond: Apache Flink in the CloudsGreg Hogan · Flink Forward Berlin 2016
• 38mApache Flink: A Very Quick GuideVladimir Bacvanski · Data By the Bay 2016
• 46mKeep it going — How to reliably and efficiently operate Apache FlinkRobert Metzger · Flink Forward Berlin 2017
• 38mFaster and Furiouser: Flink DriftTed Dunning · Flink Forward Berlin 2016
• 41mRunning Apache Flink everywhereStephan Ewen · Flink Forward Berlin 2016
• 33mFlink in ContainerlandPatrick Lucas · Flink Forward Berlin 2017