Fitting Stateful Functions into a streaming platform

on YouTube ↗27m

" summary

Nordstrom runs a Kafka-based streaming platform handling over 5,000 distinct resource changes per day and uses Prometheus for telemetry. Flink Stateful Functions become the event-driven control plane that stores snapshots to compacted topics, enforces credential expiry, and executes topic self-delete policies (TTL and log-size or rate thresholds). Demo shows credentials auto-expire, topics auto-delete, and snapshots upserted back to the metadata store, replacing many scheduled jobs and simplifying API services.

AI-written summary · about

" chapters (16)

1. 0:00Introduction
2. 0:33Nordstrom
3. 1:06Why a streaming platform
4. 3:19Platform interactions
5. 4:11Application applications
6. 4:41Telemetry
7. 6:59Stateful Functions
8. 8:07Blank Stateful Functions
9. 9:34Kafkatopic Function
10. 10:54Snapshots
11. 11:38Metrics
12. 13:37Stateful Match
13. 15:40Demo
14. 24:04Deleting topics
15. 24:41Additional platform behaviors
16. 27:01Outro