Build your own social media analytics with Apache Kafka
summary
Shows a working pipeline that pulls tweets via the Twitter timeline connector and a search source, ingests them with Kafka Connect, and processes them with Kafka Streams running on Quarkus/Kubernetes using the Streamy operator. Demonstrations include word counts with a 1‑hour window sliding every 1 minute, Jackson JSON serdes, and a sentiment pipeline using Deep Java Library with a 90% probability cutoff that emits alerts to a Twitter direct‑message sink. A quick experiment finds tweets with media average 2.8 retweets vs 0.8 without.
AI-written summary · about
chapters (10)
- 0:00Introduction and speaker intro
- 1:05What is Apache Kafka
- 5:05Kafka as event streaming platform
- 7:15Twitter API and Kafka Connect
- 9:46Kafka Streams API overview
- 11:12Demo 1: Timeline word analysis
- 20:51Demo 2: Sentiment analysis alerts
- 29:29Demo 3: Ad-hoc tweet analysis
- 34:36Future ideas and resources
- 36:20Q&A and closing
related talks
43mApache Kafka and KSQL in Action: Let’s Build a Streaming Data Pipeline!
53mApache Kafka and KSQL in Action : Lets Build a Streaming Data Pipeline!
44mProcess and analyze millions of events per second with open source tools
50mApplication development and data in the emerging world ...
41mOpen Source Social Applications with Apache Streams
01h 29mEvent stream processing using Kafka Streams
21mStreaming DataFrames: A New Way to Process Streaming Data
06mJay Kreps interview