Ceph QoS Refinements for Background Operations Using MClock
summary
Implements MClock QoS in Ceph OSDs, replaces static WPQ costs with a dynamic cost model (device random IOPS, sequential bandwidth, plus average object size per PG) and exposes three simple profiles balanced, high client ops, high recovery ops expressed as 0–1 proportions. Tests use Ceph CBT and fio on a one‑node lab (4 NVMe + HDDs). Results show balanced recovers ~8.25% faster than WPQ and high recovery ops ~50% faster, with client latencies equal or better and the slow backfill issue largely resolved.
AI-written summary · about
chapters (19)
- 0:00Introduction to MClock
- 1:06WPQ scheduler drawbacks
- 3:12MClock QoS controls
- 5:47MClock vs Distributed MClock
- 7:55Evolution of MClock in Ceph
- 10:01MClock cost modeling issues
- 11:49MClock scheduler internals
- 15:00MClock scheduling phases
- 18:33Dynamic cost calculation
- 21:24Profile optimizations
- 23:52Simplified QoS profiles
- 27:30Test setup and methodology
- 30:18Recovery performance results
- 32:04Client latency impact
- 33:30SSD recovery test results
- 34:59Latency improvements summary
- 36:09Scrubbing performance results
- 36:52Key conclusions
- 39:22Q&A session
related talks
35mMaking Ceph Fast in the Face of Failure
12mCeph QoS: How to support QoS in distributed storage system
37mDoing QoS Before Ceph Cluster QoS Is Available
38mEfficient Ceph Performance Troubleshooting in Production Using eBPF
20mPer OSD Recovery Bandwidth Control Based on dmClock
49mIntro to Ceph, the Distributed Storage System
02h 59mCeph Storage: a Caffeinated Primer
34mModernizing Ceph Deployments at CERN: CephFS and Object Storage Across Data Centres