Database Stalls, From the Ordinary to the Obscure
summary
VividCortex detects short database stalls down to 1 second by modeling Little’s Law and watching threads_running (concurrency) against queries per second (throughput); a 13‑second example shows concurrency spiking while throughput dips then rebounds. Causes found include exhausted EBS credits, MySQL query cache mutex, and Transparent Huge Pages. Recommended fixes are work‑centric metrics, inspect MySQL thread states not just processlist, add rate‑limits/queues or max_connections, prioritize requests, and use features like MySQL 8’s skip‑locked to reduce back pressure.
AI-written summary · about
chapters (22)
- 0:00Introduction
- 2:00What are stalls
- 7:46Where do stalls come from
- 11:10Workcentric monitoring
- 13:30Slowness
- 14:10Work
- 15:23System Stall
- 16:33Lock Contortion
- 16:54Stall Chart
- 17:45Log IO Chart
- 18:16Query Latency Chart
- 18:38Causation vs Correlation
- 19:09The Typical Interpretation
- 20:09The Alternative
- 23:18Back Pressure
- 24:51The Onion
- 26:13Hypothesis
- 27:13Query throughput diagram
- 28:38Fast queries
- 29:49Dealing with back pressure
- 33:09Can you eliminate stalls
- 34:16Conclusion
related talks
51mResolving MySQL Problems Quickly
58mWhy Databases Cry At Night
49m18 Ways to Fix MySQL Bottlenecks
47m17 Things Developers Need to Know About Databases
35mSlow database in your PHP stack? Don't blame the DBA!
59mWhy Databases Cry At Night
49mHELP!!1! The database is slow!!
37mDatabase Wizardry for Legacy Applications