ServiceRouter: Hyperscale Service Mesh
summary
ServiceRouter is Meta’s production service mesh using an embedded routing library (not sidecars) that routes O(10^10) requests/sec across O(10^4) services with O(10^6) L7 routers, cutting hardware by O(10^5) machines; 99% of RPCs use the library and 92% of traffic is to sharded services. It uses Apache Thrift, ZooKeeper-backed discovery, pick-2 with load pulling, load piggybacking and an automatic load balancing heuristic, plus geo locality rings and a cross-region control plane; the system ships, reduces imbalance and latency, and is OSDI’23-accepted.
AI-written summary · about
chapters (8)
- 0:00Intro
- 0:15Microservice Architecture
- 4:27Service Mesh
- 7:25ServiceRouter
- 13:33Latency Profile of Services
- 15:46Automatic Load Balancing
- 16:55Impact of Auto LB
- 18:46Cross-region Routing
related talks
17mServiceRouter: Hyperscale and Minimal Cost Service Mesh
38mRouting Your Microservices Architecture
28mHow a Service Mesh Helped Us Build Production Cloud-Native Services
35mNavigating the Service Mesh Ecosystem
47mMonolith to Microserv. at 20M+ requests per second: Latency and Scale Challenges
34mHyperRouter: Lessons Learnt from Building an L4 Load...
21mR2P2: Making RPCs first-class datacenter citizens
39mGrow with Less Pains - Meshing From Monolith to Microservices