Building Near Real-time Analytics Solution on AWS

on YouTube ↗47m

" summary

A working end to end demo wires a one‑page e‑commerce site through CloudFront into Kinesis Data Streams and Firehose, lands click events in S3, catalogs them with Glue and queries with Athena for QuickSight dashboards, with a tiny Lambda for field parsing. Claims: near real‑time ingestion with ~70 ms availability to consumers, but Firehose introduces a minimum one minute buffer for delivery. Setup uses only AWS managed services, completes in about 15–20 minutes and the dashboards update within minutes.

AI-written summary · about

" chapters (13)

1. 0:00Near real-time analytics intro
2. 1:17Why real-time analytics matters
3. 5:49Streaming on AWS
4. 16:42AWS streaming overview
5. 20:34Kinesis for streaming
6. 28:19Demo use case & architecture
7. 32:54Kinesis Data Stream setup
8. 34:00CloudFront for clickstream
9. 37:11Kinesis Firehose processing
10. 39:18S3 storage & cataloging
11. 41:24QuickSight visualization
12. 44:32End-to-end architecture
13. 46:57Closing & resources

" related talks

• 29mThe need for speed: Data streaming architecturesRajeev Chakrabarti · AWS AMER Summit May 2021
• 26mArchitecting real-time transactional data lakeMasudur Rahaman Sayem · AWS Summit Sydney 2025
• 47mBuilding Data Lake Platform Fully ServerlessAnurag Kale · DevOps Pro Europe 2024
• 39mBig Data Analytics With AWS AthenaWolfram Wingerath · code.talks 2019
• 44mBig Data with AWS: our lessons learned using managed servicesOlivier Flebus, Fanny Martin, Matthieu Bernardin · DevFest Toulouse 2017
• 37mBuild a Modern Data Platform on AWSArtsiom Yudovin · Big Data & RPA Conference Europe 2024
• 20mAI-Powered Customer Insights for Financial ServicesNeha Thakur · Conf42 Machine Learning 2025
• 27mBatching Was Yesterday: Real-Time Tracking & Analysis For 100+ Million VisitorsWolfram Wingerath, Felix Gessert · Flink Forward Global 2021