Optimising resource allocation in hyperconverged infrastructure
summary
Three case studies show concrete failure modes and fixes: Ceph filestore causes kernel hung tasks from delayed reclaim resolved by manual drop/compaction and tuning min_free_kbytes; Swift replication stalls on many 25 GB container DBs and XFS inode pressure (XFS inode ~39 GB) fixed by raising VFS cache pressure and forcing reclaim until upstream Swift patch lands; hugepage-backed VMs fail because KVM needs higher-order pages, solved by reducing 161 metadata workers and tweaking compaction thresholds and watermark_scale_factor. Observability uses ps, sar, perf (flamegraphs) and BPF; charms now set watermark_scale_factor per target and the fixes restore service but require monitoring.
AI-written summary · about
chapters (18)
- 0:00Start
- 7:08Overview
- 9:39Disaggregated Architecture
- 10:06Converged Architecture
- 10:25Hyperconverged
- 11:12Analyzing Hyper-Converged Architecture
- 12:00Service Map
- 14:13Application Behavior
- 17:14How Does the Swift Container Database Replication Take Place
- 19:44A Recurring Issue with a Hyper-Converged Node
- 20:49The Behavior of Kwemu
- 23:55How Is the Watermark Scale Factor Actually Calculated
- 26:09Sampling
- 28:31Bpf
- 30:16Reserved Host Memory
- 31:47Namespaces and C Groups Namespace
- 32:32C Groups
- 34:27Global Kernel Tunables
related talks
29mCeph, Xen, and CloudStack: Semper Melior
14mKeynote: Building and Sustaining Momentum in Open-Source Storage
24mThe Butterfly Effect of a Broken Disk: Top-Down Ceph Troubleshooting To Upstream Contribution
01h 50mKeynote
46mTurning OpenStack Swift into a High‑Load Storage
18mScaling Bare Metal Provisioning with Nova and Ironic at CERN
01h 05mDemo / Show & Tell
33mEscaping the Vendor Trap: A Journey for Migrating Legacy Infrastructure To Op...