Considerations for a large-scale low-latency system
summary
Keep user data near users and geo-replicate 2–3 data centers per region, using azure traffic manager to route to the lowest-latency site; warm and reuse TCP/SSL connections to avoid slow TCP congestion-window startup; send compact payloads with protocol buffers or bond plus compression and measure bytes and latencies; avoid heavy fan-out (fan to 40 machines adds seconds), prefer colocated data or a single-writer/optimistic concurrency model; use hedging by duplicating slow requests (Google Bigtable cut 99.9th from 1800ms to 74ms); measure, change, repeat.
AI-written summary · about
chapters (8)
- 0:00Intro
- 0:34Microsoft Development Center Norway
- 1:50Regions and where data MUST be stored
- 3:31Geo-replication within region
- 4:26TCP is a slow starter
- 5:59Data transfer
- 8:46Consider hedging
- 9:45Summary
related talks
31mPatterns of Low Latency
28mAchieving Rapid Response Times in Large Online Services
18mScaling APIs to Billions: Extreme-Scale Reliability
50mDesigning for Performance
26mTuning Your Cloud: Improve Global Network Performance for Applications
44mAvailability, Latency and Cost: Withstanding Outages
26mAdopting gRPC for Low Latency, High Bandwidth Scenarios
19mKeeping High-Traffic Systems Running When Things Go Wrong