Data processing with Kafka Streams and Spring Framework.

on YouTube ↗01h 44m

" summary

Builds a three-part system: a Python Raspberry Pi sensor (mocked in demo) → REST service → Kafka topic consumed by a Kafka Streams microservice that computes 5‑second windowed averages and emits alerts when thresholds are exceeded. Uses Kafka Streams on top of Kafka, the Spring Cloud Stream Kafka Streams binder, and local state in RocksDB backed by changelog topics. Demonstrates exactly‑once processing (processing.guarantee=exactly_once), shows late records can be dropped because time is tracked per partition, and the demo runs end‑to‑end.

AI-written summary · about

" chapters (33)

1. 0:00Introduction
2. 1:19About the speaker
3. 2:30Origin of Kafka name
4. 3:30Agenda overview
5. 5:46What is Kafka Streams
6. 7:19Use cases
7. 10:08Core features
8. 12:33Processor topology
9. 15:15Microservices integration
10. 17:56Spring Cloud Stream
11. 19:07Project planning
12. 22:25Building the REST service
13. 24:54Data model creation
14. 27:33Kafka integration
15. 30:23Kafka producer testing
16. 34:40Setting up Kafka topics
17. 38:00Streams app setup
18. 41:05Windowing concepts
19. 43:52Live coding aggregations
20. 47:33State management
21. 50:20Alert processor logic
22. 55:23Configuring bindings
23. 1:01:12Testing the workflow
24. 1:05:40Configuring transactions
25. 1:08:42Kafka transactions
26. 1:12:03Time management
27. 1:15:36Handling late data
28. 1:23:33Data skipping issues
29. 1:26:48Deduplication strategy
30. 1:29:16Stream time complexity
31. 1:32:36Exactly-once processing
32. 1:37:11Real-time commit config
33. 1:40:07Production deployment

" related talks

• 51mData processing with Kafka Streams and Spring Framework. Dmitriy Sergeev. Java Fest 2020 OnlineDmitriy Sergeev · Java Fest 2020
• 01h 29mEvent stream processing using Kafka StreamsFredrik Vraalsen · Berlin Buzzwords 2018
• 01h 11mBootiful Stream Processing!Viktor Gamov · Java Z-Day 2021
• 27mStream processing with Kafka streamsHamza Belmellouki · BlaBlaConf 2022
• 40mEvent Streaming Applications with Kafka Streams, Spring Kafka and ActuatorKo Turk · Voxxed Days Bucharest 2022
• 50mSpring Kafka beyond the basicsTim van Baarsen · J-Fall 2022
• 48mSpring for Apache Kafka the advanced featuresTim van Baarsen, Kosta Chuturkov · Spring I/O 2025
• 46mBuilding Self-Balancing Monitoring of Kafka Topic Processing Using Kafka ItselfNikita Ryanov · JPoint 2024