Application development and data in the emerging world ...
summary
Argues that stream processing becomes core application logic and shows a concrete stack: Apache Kafka as a durable pub/sub and storage layer, log compaction for compacted changelogs, and Kafka Streams as an embedded Java library with local, fault‑tolerant state that replays by resetting offsets. Demonstrates group management for transparent scaling and failure recovery, claims hundreds of thousands of msgs/sec throughput, notes Kafka Connect already has ~30 connectors, and reports deployment at thousands of companies including large LinkedIn workloads.
AI-written summary · about
chapters (12)
- 0:00Introduction and welcome
- 1:48What is stream processing
- 8:57Reactive architecture shift
- 12:30Stream processing on Kafka
- 17:22Kafka Streams library design
- 20:56Ordering and scaling in Kafka
- 25:13State management in stream processing
- 33:23Stream-table duality explained
- 41:35Simplicity in Kafka Streams
- 43:41Kafka Connect for data integration
- 45:31Future of Kafka Streams
- 46:14Q&A session
related talks
45mApplications in the Emerging World of Stream Processing
06mJay Kreps interview
55mDemystifying Stream Processing with Apache Kafka
03h 00mStream Processing, Going Beyond Pub/Sub with Kafka Streams and...
03mInterview
30mArchitecting Applications With Multiple Open-Source Big Data Technologies
01h 00mPutting Apache Kafka to Use
46mEmbracing Anarchy