Caching for Agentic Java Systems: Internal, Distributed, and Semantic
summary
Covers concrete patterns and numbers: use internal Caffeine cache for nanosecond reads, combine with distributed Valkey/Redisson for shared state (client‑side invalidation, per‑key TTL, three‑line RedisRateLimiter), scale to ~100 nodes with up to 500 GB per node and four replicas, and use semantic caching with Valkey Search (HNSW vector search, 2–4k dimensions ≈16 KB/vector) to avoid expensive LLM inference by vectorizing queries and returning cached responses above a similarity threshold (examples: 0.94 vs 0.75).
AI-written summary · about
chapters (12)
- 0:00Caching in Java systems
- 1:05Caching approaches survey
- 3:55Internal caching with Caffeine
- 7:10Distributed caching with Redis/Valky
- 13:37Rate limiting with Redis
- 21:36Semantic caching concepts
- 28:13Vector similarity search methods
- 32:26Vector search trade-offs
- 36:19Semantic caching workflow
- 41:13Valky for vector search
- 44:58Semantic caching demo
- 47:44Summary & Q&A
related talks
44mHow to speed up your application using JCache
45mCaching in Applications Still Matters
29mUnleashing the Power of Vector Search and Semantic Caching in Java
01h 10mCaching with Spring: Advanced Topics and Best Practices
45mCaching Unleashed: Supercharge Your Database Application
51mSecrets of Agentic Memory Management
31mGotta Cache 'em All: Scaling AI Workloads With Model Caching in a Hybrid...
44mCaching Unleashed: Revolutionizing Java Performance