Ceph Performance Tuning: From Bluestore to RBD
summary
Uses CBT Upstream and fio/hsbench for repeatable tests on a 60‑OSD NVMe cluster and shows concrete wins: reads hit 4.5 million IOPS while burning ~8 cores per OSD; raising PG count 4096→16000 raises reads from ~3.1M to ~4.5M. RBD snap‑trim fixes cut OSD CPU from ~250% down to ~1 core; RocksDB TTL compaction drops insertion latency from ~60,000 ms to ~10,000 ms and disk use from ~90–95% to ~20–25%.
AI-written summary · about
chapters (14)
- 0:00Introduction and setup
- 1:06Is Ceph slow?
- 2:55Benchmarking tools overview
- 5:00When Ceph performs best
- 6:45SSD firmware impact on performance
- 8:12PG count and performance scaling
- 11:44MDS directory distribution
- 13:09IO500 benchmark insights
- 15:57RBD snap trimming CPU usage
- 19:30OSD threading efficiency
- 21:19PG log rocksdb interaction
- 24:31RocksDB tombstone compaction issue
- 30:50Summary and key takeaways
- 31:53Q&A session
