Resilient Design 101: Queue Theory
summary
Wix runs ~500 microservices for ~100 million users and queue theory explains why latency explodes as utilization nears 100%: small variance makes queues grow toward infinity, so provision to keep utilization around 50–70% and limit queues (socket backlog default 128). Enforce back pressure (HTTP 503), avoid unbounded callback queues, and remember Kafka has no built‑in backpressure. Tune thread pools (Java thread stack ~1 MB), use Little’s Law to infer hidden queue sizes, and pick timeouts from measured histograms and human thresholds (20 ms, 100 ms, 1 s).
AI-written summary · about
chapters (19)
- 0:00Intro
- 3:36Queues
- 8:09Limit Queues
- 11:51In practice
- 12:39Backpusher
- 13:55Blocking
- 15:12Loadshedding
- 16:35Queue Architecture
- 17:55How many threads
- 19:56How to optimize
- 22:31Synchronous systems
- 25:41Synchronous overload
- 26:33Synchronous servers
- 27:28Littles Law
- 30:35Example
- 32:29Timeouts
- 35:24RealWorld Constraints
- 37:33Timer Budgets
- 39:05Debt Buyer
related talks
40mSystem Performance and Queuing Theory - Concepts and Application
40mEverything Will Flow
38mA Practical Look at Performance Theory
30mThe Queue Continuum: Applied Queuing Theory
44mDeep Dive talk | Resilience in Microservices with Congestion Control
44mScaling Horizons Effective Strategies for Wix's Scaling challenges
29mResilient by Design
06mQueuing Theories