In Depth Kafka Streams
summary
Shows Kafka Streams plus KSQL doing real work not theory: producers write keyed messages into topics with three partitions in the demo, consumers rebalance one-per-partition, retention defaults to 168 hours, and a streams app groups orders by state and maintains running counts backed to RocksDB while applying compaction so only latest totals remain. Demonstrates tumbling, hopping and session windowing, filters that route California orders to a separate topic, Avro payloads, and how a KSQL persistent query creates a new topic versus nonpersistent ad hoc queries, all working live with subsecond downstream updates.
AI-written summary · about
chapters (14)
- 0:00Start
- 5:58How Kafka Works Internally
- 9:37Kafka Message
- 20:14Compaction
- 21:20Why Compaction
- 22:15Clean and Dirty
- 30:47Windowing
- 31:38Hopping Windows
- 32:15Session Ization
- 42:42State Orders Count
- 44:50K Sequel
- 50:23What Is a Persistent Query versus a Non Persistent Query
- 55:01Persistent Query
- 57:44Conclusion
related talks
01h 07mKafka Streams and KSQLDB
03h 00mStream Processing, Going Beyond Pub/Sub with Kafka Streams and...
39mKafka as a Platform: The Ecosystem from the Ground Up
43mApache Kafka and KSQL in Action: Let’s Build a Streaming Data Pipeline!
53mApache Kafka and KSQL in Action : Lets Build a Streaming Data Pipeline!
39mAn introduction to Apache Kafka - this is Event Streaming, not just Messaging
01h 29mEvent stream processing using Kafka Streams
55mReal-Time Data Analysis with Apache Kafka