D4N in RGW
summary
D4N is an intelligent RGW cache that is being upstreamed into Ceph and runs as a Zipper API filter with four drivers: filter, policy, directory, cache backend. Current code supports an SSD backend and a Valkey/Redis directory, LFUDA (and LRU) replacement, and a single-node read cache already merged; single-node write‑back (with head‑block caching, dirty‑block cleaning thread, and ordered version commits) is under upstreaming. Research adds the Kariz prefetcher for DAG‑based prefetching and aims for distributed, partitioned cache operation.
AI-written summary · about
chapters (9)
- 0:00Introduction to D4N
- 1:02What is D4N cache
- 3:30Upstreaming D4N in RGW
- 6:47High-level design
- 9:24Read cache details
- 11:30Write-back cache details
- 14:21Upstreaming status
- 16:07Use cases for D4N
- 17:33Q&A
related talks
36mUsing NGINX as a CDN for Ceph Radosgw
13mQuick Cache DSR
38mExploring Distributed Caching for Faster GPU Training with NVMe, GDS, and RDMA
19mP2CACHE: Exploring Tiered Memory for In-Kernel File Systems Caching
14mSmarter RAG, Smaller Bill: Optimize for Performance and Price
43mCaching half a billion user transactions
35mEverything You Wanted to Know About RadosGW
22mAccelerate the Image Distribution for DADI Using Intel DSA