Configuring Small Ceph Clusters for Optimal Performance
summary
Three‑node Ceph cluster (40 Gb NICs) tests show out‑of‑the‑box defaults underperform and small clusters need tuned PG counts, balancing and sizing. Using balancer plugin or manual CRUSH weight changes helps but the best wins come from increasing PGs: 128→256 doubles moves ~35% of data and drops STD‑dev of PG distribution (e.g. ~1.8→1.4), boosting throughput from ~750 MB/s toward ~1,250 MB/s. Watch RGW objecter queue and memory—raising object size improves IO but causes container swap when RGW hits ~8 GB. Use PG autoscaler and Nautilus features to control changes.
AI-written summary · about
chapters (12)
- 0:00Introduction to small Ceph clusters
- 1:35Why small Ceph clusters need tuning
- 4:12Identifying bottlenecks in Ceph
- 8:04Data balancing and weakest link theory
- 13:14Balancing rules and PG distribution
- 18:54Ceph balancer tool overview
- 23:51PG autoscaler and performance impact
- 29:44Checking cluster balance with Ceph tools
- 33:16Impact of changing PG count
- 36:08Bottlenecks beyond storage devices
- 38:58RGW memory and swapping issues
- 40:22Summary and optimization tips