Crunching data in real time for Fortune500 clients
summary
Real-time personalized content delivery for Fortune500 sites (L Oreal, Unilever, Purina, Nutella) uses a measurement-driven, distributed design: geoDNS (Route 53) routes users to nearest region, two US datacenters reduce coast-to-coast latency, databases use master-master Galera cluster with odd-numbered quorum nodes, background processing via queues (0MQ/Rabbit/Kafka) and real-time workers in memory (Redis) next to user-facing servers; system prioritizes availability over strict consistency and runs at scale for hundreds of millions monthly.
AI-written summary · about
chapters (13)
- 0:00Introduction
- 2:25Client list
- 3:03The problem
- 4:00The basics
- 7:35Cap Theorem
- 13:10Replication
- 17:30Availability
- 21:20Scaling
- 25:10Decouple
- 27:45Realtime
- 31:35User Architecture
- 33:00Summary
- 35:25Questions
related talks
34mHow We Handle Over 1 Trillion HTTP Requests ...
29mRealtime Analytics Unleashed: Unlocking the Potential of Data at Scale
42mUsing Data Orchestration for speed, reliability and real-time content
08mReal-Time Data Systems: Speed, Scale, and Resilience
28mMoving Towards Real-Time OLAP - With First Principle Thinking and Guided by Access Patterns
45mRuby for the soul of Big Data nerds
17mA Real Time Analytics Framework
21mBuilding Distributed Platforms at Extreme Scale on Fast Changing Data