Reducing gRPC Call Volume Through Caching and Batching
summary
Netflix wires cache directives into protos and injects interceptors so Java clients use Google/Guava and off‑heap caches and a distributed Netflix/EVCache without API changes; result: fleetwide Java-client cache hit rate is ~75% so only ~25% of calls reach servers, and a particular experimentation path drops to ~5% hitting servers. An in-channel interceptor buffers messages, non‑blocking queries caches, and on cache miss forwards to transport. A separate request batching implementation aggregates nearby stub calls into one RPC but is now rare and slated for retirement; streaming and security contexts complicate caching.
AI-written summary · about
chapters (12)
- 0:00gRPC caching and batching intro
- 1:43Caching expensive gRPC ops
- 3:28Proto-based caching config
- 6:36Advanced caching config
- 8:42Caching limits and security
- 9:45Caching strategies
- 10:27Measuring Netflix caching
- 11:53Request-scoped caching
- 12:55Inefficient call patterns solved
- 14:18gRPC request batching details
- 15:43Batching evolution and retirement
- 16:26Q&A
related talks
28mHow Netflix Makes gRPC Easy to Serve, Consume, and Operate - Benjamin Fedorka, Netflix
35mCaching at Netflix: The Hidden Microservice
22mHow Netflix Leverages gRPC for Resilience at Scale
47mArchitectural Patterns for Caching Microservices
55mgRPC 101 for Java Developers
50mEnabling Googley microservices with gRPC
46mIntroduction to gRPC
53mHow Netflix Uses Java - 2026 Edition