Analyzing streaming data with Apache Druid

on YouTube ↗20m

" summary

Apache Druid ingests Kafka/Kinesis/MSK streams natively with true stream ingestion and guarantees exactly‑once delivery, placing events into the database and S3 continuous storage so queries see late arrivals correctly. Typical query availability falls around 10–20 ms, with sub‑second analytic queries at high concurrency (hundreds per modest server) and production clusters reported up to 15,000 nodes / 400 PB. Used for operational visibility (Atlassian 3 billion events/day), observability (ThousandEyes), ad decisioning (Reddit) and IoT, plus a Polaris cloud service and hybrid managed options.

AI-written summary · about

" chapters (8)

1. 0:00Analyzing streaming data
2. 2:16Real-world use cases
3. 4:26Key Druid applications
4. 5:58Streaming challenges
5. 9:33Why choose Druid
6. 12:23Druid history and use
7. 14:07Introducing Imply
8. 17:22Imply on AWS and resources

" related talks

• 46mScalable Realtime Analytics using DruidNishant Bangarwa · The Fifth Elephant 2016
• 39mApache Druid 101Matt Sarrel · Data Con LA 2020
• 28mBest of Stream + Batch: When Streaming Analytics Isn’tRohan Garg · DataEngBytes 2022
• 45mDruid: Powering Interactive Data Applications at ScaleFangjin Yang · Strange Loop 2016
• 27mSucceeding with Apache Druid and Clickstream DataPeter Marshall · OSA Con 2021
• 20mReal-time analytics with Flink and Druid #bbuzzJan Graßegger · Berlin Buzzwords 2016
• 41mBuilding an Open Source Streaming Analytics Stack with Kafka and DruidFangjin Yang · Embedded Linux Conference + OpenIoT Summit North America 2017
• 28mThe Heat is On: Architecting for Hot AnalyticsGian Merlino · Data Love 2021