Linux IO internals for PostgreSQL administrators in 2023
summary
Postgres moves pages from disk into the Linux page cache and then copies them into shared buffers, so writes mark whole pages dirty and checkpoint spikes are the usual source of large IO bursts; WAL must be fsynced for durability and Postgres currently supports direct AIO only for WAL writers. Modern kernels use blk-mq/mq-deadline (blk-mq replaces the old elevator) and NVMe parallelism and polling improve throughput, but huge RAM plus generic EBS can still saturate disks. Recent kernels and testing with pg_stat_io or perf are recommended.
AI-written summary · about
chapters (11)
- 0:00Introduction to Linux IO
- 1:52Why Linux IO matters
- 4:02PostgreSQL IO stack
- 9:01Dirty pages and checkpoint spikes
- 13:15IO bottlenecks and SSD era
- 16:13Linux IO stack evolution
- 20:52Blk-mq and modern schedulers
- 27:19NVMe and database readiness
- 30:31Direct IO in PostgreSQL
- 33:23Further learning resources
- 35:56Q&A session
related talks
01h 06mIO in PostgreSQL: Past, Present, Future
49mAIO in PG 18 and beyond
30mPostgreSQL and Linux Kernel interactions
53mVisualizing Postgres I O Performance for Development
26mAdditional IO Observability in Postgres with pg_stat_io
48mStreaming I/O and vectored I/O
49mStreaming I/O and vectored I/O
26mLinux and PostgreSQL in the Multiverse of Connections