How to run Real Time processing on Big Data
summary
Describes a RAM-first architecture that layers an in-memory data grid over SSDs so hot data serves microsecond reads while cold data persists to disk, claiming memory access is ~1000x faster than disk. Integrates with Cassandra + SSD tiering, supports delta updates, projections, streaming queries and multi-master replication with automatic scaling. Deployed for high-traffic e-commerce (survives Black Friday) and a bank running roughly 100 clusters, reducing outages and storage costs in production.
AI-written summary · about
chapters (8)
- 0:00Introduction
- 3:29Real-time data processing challenges
- 6:58Traditional architecture limits
- 13:10In-memory solution
- 22:03Big data integration and analytics
- 27:50Use cases and benefits
- 31:48Customer cases and results
- 34:12Q&A
related talks
43mArchitecting for Speed & Scale
08mReal-Time Data Systems: Speed, Scale, and Resilience
48mDesign of data-intensive applications
45mModern Data Architecture
40mArchitecting for Data-Intensive Workloads
10mData Rivers — The New Analytics Architecture
38mRealtime Data Activation - Analytics, Intelligence & Decision Making
12mGetting started with fast data architecture