Why Databases Cry At Night
summary
Backups fail because dozens of tenant databases flood network storage and cause timeouts — in one case >200 concurrent backups overload the SAN queue; the fix is serializing backups or provisioning IOPS. Low-level tools used: Process Monitor and CrystalDiskMark (and an ATTO-style benchmark) to measure random IOPS and write‑through performance. An LMDB experiment writes 100 transactions × 100 keys showing sequential keys require far fewer seeks and lower commit latencies. Notes: LSM compactions (Cassandra) cause intermittent read slowdowns, WAL write‑through is costly and often benefits from a dedicated fast disk.
AI-written summary · about
chapters (18)
- 0:00Start
- 2:16The Law of Leaky Abstractions
- 3:40Here is a riddle... hint: storage!
- 9:04What can we do about storge issues?
- 13:47A tale of two primary keys
- 17:54Storage algorithms (page-oriented storage engines)
- 18:41B+ tree keys
- 20:00Minimize performance impact of keys • Sequential keys allow better performance
- 22:11Storage algorithms (log-structured storage engines)
- 24:58Storage algorithms - LSM Tree compaction
- 26:51Different databases have specific options to optimize performance For example
- 30:01Write-ahead log (WAL) - Atomicity, Durability Writing
- 34:36ATTO Disk Benchmark
- 34:56Buffered vs. Write-through
- 36:11Storage effect on transactions
- 46:44What can (should!) we do?
- 50:32Indexing complexity
- 54:01To sum it up
related talks
11mThe Brave Little Database!
42mWhy databases cry at night?
35mQuestioning Database Claims: Design Patterns of Storage Engine
29mMySQL Performance for DevOps
35mDatabase Stalls, From the Ordinary to the Obscure
34mWhat We Talk About When We Talk About Disk IO
57mDSS Keynote — Back to the Future: Prepare Your Data Infrastructure for Anything
40mLatency, Throughput & Fault Tolerance: Building a Modern Database