Define data streaming architectures & solutions to fit your needs

on YouTube ↗53m

" summary

Overview of AWS streaming choices with concrete tradeoffs: Amazon Kinesis Data Streams + Data Firehose for simple managed ingestion and delivery (now Kinesis Data Firehose supports direct sinks and up to a year retention) vs. Amazon MSK (managed Apache Kafka) and MSK serverless for lower-latency, offset-friendly workloads. Kinesis shows about 200 milliseconds end-to-end typical latency; some Kafka customers push toward ~20 ms. Schema Registry, MSK Connect (MirrorMaker patterns), replay via offsets, and scaling/partition cooldowns are demonstrated.

AI-written summary · about

" chapters (21)

1. 0:00Intro
2. 1:30What is data streaming
3. 5:05Use cases
4. 9:38Data streaming
5. 13:50Amazon Kinesis
6. 14:47Kinesis Data Streams
7. 16:14Apache Link
8. 18:03Apache Kafka Managed Treatment
9. 19:20Apache Kafka Connect
10. 20:18Schemas
11. 24:58Kafka vskinesis
12. 27:28Region replication
13. 32:48Latency
14. 33:53Network
15. 35:28Offsets
16. 36:48Glue vs Apache Flink
17. 40:06Errors in data streams
18. 42:23kinesis vs Kafka
19. 46:55Scaling operations
20. 48:26cooldown period
21. 49:18limitations

" related talks

• 29mThe need for speed: Data streaming architecturesRajeev Chakrabarti · AWS AMER Summit May 2021
• 35mStreaming and Real-time AnalyticsDonnie Prakoso · AWS Summit Online EMEA 2020
• 35mStreaming and Real-Time AnalyticsDonnie Prakoso · AWS Summit Online 2020
• 23mCustomer Journey with streaming data on AWSRahul Pathak · Flink Forward Europe 2019
• 55mCollecting Big Data at Scale with Amazon KinesisRohit Bhardwaj · Developer Summit 2020
• 39mLarge scale streaming analyticsJay Palaniappan · Data Con LA 2019
• 26mArchitecting real-time transactional data lakeMasudur Rahaman Sayem · AWS Summit Sydney 2025
• 44mData Streaming: Which horse to bet on?Kamil Owczarek · GeeCON 2019