Enriching Kafka Applications With Real-time Contextual Data

on YouTube ↗49m

" summary

Claim: enrich Kafka streams by co-locating fast storage and compute so real-time events join with historical context with millisecond-scale latency. Demo uses Kafka + Hazelcast running on a laptop in embedded mode (also notes client and pipeline options), shows a Java pipeline and SQL join with in-memory map that flags “red” fraud events, counts windows per second, and outputs to another topic. Typical banking reply target is ~50–100 ms; the demo runs end-to-end and validates immediate enrichment without an external DB.

AI-written summary · about

" chapters (6)

1. 0:00Introduction to real-time Kafka
2. 10:05Batch vs streaming processing
3. 21:19Is Kafka enough for apps?
4. 31:36Unified platform benefits
5. 37:43Demo Kafka enrichment
6. 47:51Key takeaways and Q&A

" related talks

• 19mSupercharging Kafka Applications with Real-time Contextual InsightsRaj Barua · Great International Developer Summit 2024
• 30mFive Challenges in Real-time Stream Processing and Five SolutionsFawaz Ghali · SREday 2023 London
• 01h 00mIn Depth Kafka StreamsDaniel Hinojosa · Devnexus 2020
• 50mApplication development and data in the emerging world ...Neha Narkhede · Berlin Buzzwords 2016
• 43mApache Kafka and KSQL in Action: Let’s Build a Streaming Data Pipeline!Robin Moffatt · LISA18
• 53mApache Kafka and KSQL in Action : Let’s Build a Streaming Data Pipeline!Viktor Gamov · Jfokus 2019
• 45mApplications in the Emerging World of Stream ProcessingNeha Narkhede · GOTO Chicago 2016
• 06mJay Kreps interviewJay Kreps · Data By the Bay 2016