The need for speed: Data streaming architectures
summary
Modernize a lakehouse by ingesting high‑velocity sources into Amazon Kinesis Data Streams or Amazon MSK, retain streams up to one year and process with Lambda, Kinesis Data Analytics (SQL or Apache Flink), or Spark on EMR. Zillow cuts Zestimate compute time from hours to seconds by feeding Kinesis → Spark and Firehose → S3; Netflix ingests multiple terabytes daily using ~100 Kinesis shards to surface flow‑log insights in seconds. Glue Schema Registry enforces schemas serverless and free, reducing downstream breakages.
AI-written summary · about
chapters (15)
- 0:00Start
- 1:01The volume of data being produced is increasing
- 2:44The value of data diminishes over time
- 4:58Common real-time analytics use cases
- 7:24Achieving real-time analytics use cases
- 8:10Challenges of data streaming
- 8:59Streaming real-time data with AWS
- 11:44Data sources
- 11:57Stream ingestion
- 12:25Stream storage
- 13:15Kinesis Data Streams
- 14:35Amazon MSK
- 15:54AWS Glue Schema Registry
- 18:52Stream processing
- 28:12Learn data analytics with AWS Training and Certification
related talks
35mStreaming and Real-time Analytics
53mDefine data streaming architectures & solutions to fit your needs
35mStreaming and Real-Time Analytics
29mSimplifying data flows
23mCustomer Journey with streaming data on AWS
55mCollecting Big Data at Scale with Amazon Kinesis
39mLarge scale streaming analytics
43mManaging a Data Lake on AWS