299,792,458 Meters per Second
summary
Network latency shapes design decisions: ~14.8 million meters gives a 50 milliseconds theoretical round trip, but observed pings hit 250 ms measured ping between Chicago and Sydney. Setup uses five to six data centers and mixes strategies: MySQL/Postgres logical replication (Postgres 10 caveats), queues (Rabbit/Kafka patterns), and a homegrown WebSocket fan‑out that populates an edge cache with memcache edge cache. Operational tooling includes Datadog, centralized logging and PagerDuty; the homegrown system works but they plan replacement.
AI-written summary · about
chapters (32)
- 0:00Intro
- 2:29HOW OUR SYSTEMS OPERATE MOST OF THE TIME
- 3:54GEOGRAPHICALLY DISTRIBUTED SYSTEMS PRESENT UNIQUE CHALLENGES
- 5:08THE IMPACT OF NETWORK LATENCY
- 5:23THE MATHS
- 5:54CHICAGO TO SYDNEY
- 7:39REAL WORLD
- 14:58NETWORK LATENCY IS REAL
- 15:22MOVING DATA TO THE EDGE
- 15:46BENEFITS
- 17:12DRAWBACKS
- 18:09FROM SINGLE DATA CENTER TO MULTIPLE DATA CENTER
- 19:04ENDING STATE
- 19:23UNDERSTANDING YOUR REQUIREMENTS
- 22:23IMPLEMENTATION OPTIONS
- 23:01RELATIONAL DATABASES
- 25:13SYNCHRONOUS VS. ASYNCHRONOUS REPLICATION
- 34:07MESSAGE QUEUES + KEY-VALUE STORE
- 34:56REPLICATION?
- 37:15WHAT WE'VE USED AT DNSIMPLE
- 39:26TRANSPORT SECURITY
- 39:53FIREWALLS
- 40:13ACCESS CONTROL
- 40:52OPERATIONAL CONSIDERATIONS
- 41:45AGGREGATED LOGGING
- 42:52INCIDENT MANAGEMENT
- 43:37DISTRIBUTED TRACING
- 45:20CONCLUSION
- 46:16THERE ARE MULTIPLE OPTIONS
- 46:36VERIFICATION
- 47:23IMPLEMENT TECHNOLOGY YOU KNOW IF IT FITS YOUR REQUIREMENTS
- 48:04QUESTIONS?
related talks
43mDistributed Systems
47mMonolith to Microserv. at 20M+ requests per second: Latency and Scale Challenges
38mTechniques and Tools for a Coherent Discussion about Performance in Complex...
01h 28mHigh Performance Stream Processing
32mBuilding distributed system for retail with RabbitMQ
24mBuffalo: when Node and APIs become BFFs
37mHow Geo-Distributed Apps Use Global Cloud Infra
23mAdmiral Scales Globally with YugabyteDB on GCP + Single-Digit Latency