Building a Telegram bot with Apache Kafka and ksqlDB

on YouTube ↗47m

" summary

Builds a Telegram bot that answers “how many spaces” and sends alerts by ingesting a CSV parking API into Kafka (using kcat in Docker, polling ~3 minutes), modeling events with ksqlDB to create a materialised view for key/value lookups, and triggering event-driven alerts (example alert set for 500 spaces). Uses Protobuf and a protobuf schema in Confluent Schema Registry, pushes the same stream to Elasticsearch via Kafka Connect for dashboards; demo mostly works though the live API is flaky.

AI-written summary · about

" chapters (6)

1. 0:00Start
2. 0:34Building a Telegram bot
3. 35:04Event Driven Alerts
4. 35:27KN Lookups (materialised views)
5. 36:13A schema...
6. 36:51Integration

" related talks

• 26mBuilding a Telegram bot with Apache Kafka and ksqlDBRobin Moffatt · DataEngBytes 2020
• 43mApache Kafka and KSQL in Action: Let’s Build a Streaming Data Pipeline!Robin Moffatt · LISA18
• 53mApache Kafka and KSQL in Action : Let’s Build a Streaming Data Pipeline!Viktor Gamov · Jfokus 2019
• 31mPractical Pipelines: A Houseplant Alerting System with ksqlDB.Danica Fine · Scale By the Bay 2023
• 01h 05mPractical Pipelines: A Houseplant Soil Alerting System with ksqlDBDanica Fine · NDC Oslo 2022
• 44mNo More Silos: Integrating Databases and Apache KafkaRobin Moffatt · code.talks 2019
• 39mKafka as a Platform: The Ecosystem from the Ground UpRobin Moffatt · GOTOpia Europe 2020
• 41mKafka as a Platform: The Ecosystem from the Ground UpRobin Moffatt · GOTOpia November 2020