Building Scalable Caching Systems via McRouter
summary
McRouter is a memcached protocol router that Facebook, Instagram and Reddit use in production; Facebook routes over four billion cache ops per second and Instagram over 100 million, and the project is open source today. It implements connection pooling to reduce O(nT) TCP connections to O(n), prefix-based pool partitioning for heterogeneous workloads, automatic failover* with periodic probes and delete spooling for consistency, cold-cache warmup, shadow testing; Reddit integrates via ZooKeeper and Ubuntu packages and validates at a 4,200 ops/s cluster before scaling up.
AI-written summary · about
chapters (20)
- 0:00Intro
- 0:44The Social Graph
- 1:28Dynamically Rendering the Graph
- 1:54Caching @ Facebook
- 2:42Caching @ Instagram
- 3:27Cache is a performance necessity
- 3:37No longer just an optimization
- 4:01pre mcrouter
- 4:51Enter mcrouter
- 6:48Connection Pools Problem Too many connections can Overload a server
- 7:15Heterogeneous Workloads
- 7:59Automatic Failover
- 9:31Common Evolutionary Path
- 10:15reddit infrastructure
- 11:01Testing new instance types takes time Performance issues will be visible to users
- 11:21Pool modification requires downtime
- 11:52Reheat time 16 hours and counting...
- 12:22Integration
- 13:12Not a silver bullet
- 14:02The Future
related talks
23mScaling Memcache
33mWhat the Cache?!
15mBuilding Self-Healing Cache Infrastructure
31mFifty Shades of Caching and How LLMs Paint It Blαck
49mUnlocking Performance with Response Caching
34mAdvanced Caching Patterns used by 2000 microservices
25mCaching strategies on https://dev.to
42mPowering 100+ million daily users