Rethinking Writeback: Scaling Linux Filesystem and Memory Performance for the Next Decade
summary
Proposes replacing the single threaded Linux writeback with multiple BDI writeback contexts that assign each inode to one thread (inode % contexts) and optionally to an XFS allocation group (per-AG assignment) to avoid AG lock contention. Implements on-the-fly tuning via a sysfs-like attribute that freeze/thaws the filesystem, benchmarks with fio and a Vector DB and sees up to ~5x IOPS improvement and device saturation near 2,000 MB/s (baseline ~365 MB/s); kernel build drops ~10s, HDDs can regress ~14%.
AI-written summary · about
chapters (24)
- 0:00Parallel writeback research intro
- 1:03Buffered I/O and writeback stack
- 7:21Single-threaded writeback bottleneck
- 8:04Parallelizing writeback design
- 10:54Multiple BDI writeback contexts
- 13:00Delayed allocation impact
- 14:47Distributing I/O to threads
- 17:44Allocation group lock contention resolution
- 18:04Configurable writeback threads
- 20:12Memory and CXL impact
- 21:44GenAI workload benefits
- 23:07Vector DB benchmark eval
- 23:47FIO results and improvements
- 24:30AG-based writeback perf
- 25:12Kernel compile time reduction
- 25:53Acknowledgments and conclusion
- 26:17Q&A
- 27:01Q&A prioritization in parallel writeback
- 28:06Q&A debugging complexity
- 30:37Q&A AG usage impact
- 33:06Q&A tuning writeback threads
- 35:16Q&A hardware compatibility
- 36:19Q&A ext4 performance
- 36:59Q&A thread overhead
related talks
16mRearchitecting Buffered I/O in the Era of High-Bandwidth SSDs
24mOptimizing Storage Performance for 4–5 Million IOPs
14mAdvancing Data Integrity in Linux
23mImproving data placement for Zoned Linux File systems
45mLinux Storage Stack Explained
43mThe Forefront of the Development for NVDIMM on Linux Kernel
47mExploring Linux Memory Usage and IO Performance in Cloud Native Applications
15mRearchitecting Linux Storage Stack for µs Latency and High Throughput