Apache Flink Worst Practices
summary
Concrete anti-patterns and concrete fixes: decide upfront whether you need at-least-once or exactly-once semantics because exactly-once requires replayable sources, checkpoints and transactional sinks and blows up latency for sub‑200ms SLAs; plan upgrades by assigning UIDs to operators and using savepoints; avoid Kryo and deep JsonNode objects—use POJOs/Avro and simple key types; estimate state = keys_per_task × object_size and include checkpoint I/O in capacity planning; don’t use static vars or spawn threads, initialize in open(), prefer filesystem state for fast setups and only move to RocksDB after measurement.
AI-written summary · about
chapters (16)
- 0:00Intro
- 2:07Before you start...
- 3:07Training & Community
- 6:28Consistency & Delivery Guarantees
- 9:22Application Evolution
- 15:20Feasibility Check
- 19:50Business Requirements
- 24:36SQL or DataStream API
- 27:02Data Types
- 29:51Serialization ctd.
- 31:25Concurrency
- 33:57Windowing
- 35:45Data Stream API Classics
- 37:03Flink's Test Harnesses
- 37:22Go Live
- 41:25Maintenance
related talks
42mApache Flink Worst Practices
28mIt's Time To Stop Using Lambda Architecture
26m“Hit me, baby, just one time” – Building End-to-End Exactly-Once Apps w/ Flink
42mApache Flink deep-dive #bbuzz
29mManaging State in Apache Flink
36mApache Flink in the Cloud-native Era
47mTop 10 Event-Driven Architecture Pitfalls
38mFaster and Furiouser: Flink Drift