Queueing Theory in Practice: Performance Modeling for the Working Engineer
summary
Small experiments measure one core’s latency under load (thousands of req/s, sub-millisecond latencies) and fit a simple M/M/1-style model with a representative 200 µs service time to predict the utilization “knee” accurately. The talk shows the model matching real test data, explains why variability worsens queues, quantifies coordination cost with the Universal Scalability Law, and demonstrates practical fixes that work: two-choice load balancing and tree-structured aggregation cut coordination overhead and restore useful scaling.
AI-written summary · about
chapters (8)
- 0:00Introduction to theory
- 2:44Modeling serial systems
- 6:13Analyzing wait times
- 11:15Performance lessons
- 15:19Introduction to parallel
- 15:50Universal scalability law
- 20:15Scalability strategies
- 23:32Conclusion and takeaways
related talks
30mQueueing Theory, In Practice: Performance Modelling in Cloud-Native Territory [I]
06mPractical performance theory
38mA Practical Look at Performance Theory
30mWhat's the Cost of a Millisecond?
45mThe Universal Laws of Performance
50mDesigning for Performance
45mUnderstanding Microservice Performance
44mActive Queue Management - What it is, and why you should care