Adaptive Concurrency Control for Mixed Analytical Workloads
summary
Klaviyo’s real-time Query Service hits intermittent waves of timeouts despite adequate capacity; investigation shows queuing and variable query latency are the root cause. Implementation adds Python gRPC/Envoy-level per-server in-flight limits, samples P95 latency over rolling windows and runs an AIMD (additive increase, multiplicative decrease) controller to raise or sharply back off the concurrency ceiling, and enforces partitioned quotas so API, UI and reports keep reserved slots. The system leverages ClickHouse, client-side opaque retries, and turns timeout walls into short latency spikes while improving cluster utilization.
AI-written summary · about
chapters (9)
- 0:00Start
- 7:28Metrics Service Request Flow - gRPC Server
- 9:20Healthy State - Queuing Balanced with Processing
- 15:54Load Shedding at the Cluster Level
- 18:31Load Shedding and Concurrency Control
- 20:25Adaptive Concurrency Control - Measuring Latency
- 21:22Adaptive Concurrency Control - Record, Recalculate, React
- 23:11Adaptive Concurrency Control - AIMD Algorithm
- 25:03Adaptive Concurrency Control - Partition Limits
related talks
32mTackling Slow Queries: A Practical Approach to Prevention and Correction
27mReducing Client Latency and Timeouts When Running Cassandra on Public Cloud
28mMoving Towards Real-Time OLAP - With First Principle Thinking and Guided by Access Patterns
24mDevOps. Monitoring Databases. On-Premise and Cloud environments
29mScaling Rails to 2M MySQL requests per second
54mDistributed Caching in Real-Time Payment Systems
35mDatabase Stalls, From the Ordinary to the Obscure
22mIt’s all about the db - a journey to accommodate 100k events per second