Real-time analytics at IoT scale
summary
Rivian runs a cloud streaming stack on AWS EKS that ingests telemetry into Kafka, Flink, Druid and persists segments to S3, claiming sustained 2 million events/sec with average <100 ms latency. Configurable “event watch” lets teams deploy SQL or pattern rules, supports staleness, deduplication and cross-event state (pet‑mode + low charge), and custom pipelines like geofence detection that bucket with geohash to avoid full scans. Backfill and reprocessing come from the Delta Lake.
AI-written summary · about
chapters (13)
- 0:00Introduction
- 1:10Electric vehicle product line
- 2:38Vehicle architecture data needs
- 4:05Data platform for real-time analytics
- 6:11Stream processing with Flink
- 10:48Flink deployment and pipeline design
- 13:39Stream processing pipeline anatomy
- 16:07Geofence detection pipeline
- 19:38Event Watch service
- 23:57Query service with Druid
- 28:14Druid ingestion from Kafka
- 31:52Druid on Kubernetes scaling
- 33:22Q&A
related talks
30mData Platform Infrastructure at Rivian: A Deep Dive
20mAnalyzing streaming data with Apache Druid
39mLarge scale streaming analytics
38mRealtime Data Activation - Analytics, Intelligence & Decision Making
50mThe Real-Time Analytics Stack
35mStreaming and Real-Time Analytics
22mTurn Data Into Insights
28mTransforming Automotive Infotainment with Data