Troubleshooting a Cephadm Deployment
summary
Cephadm troubleshooting focuses on getting actionable logs not just “scheduled” messages: enable per-service and cluster log files, raise log_level 5 (default 1 is useless), and capture container-internal commands with Snoopy logging (auditd spams logs and SSH ForceCommand and PROMPT_COMMAND don’t capture noninteractive shells). Verify prerequisites (Python, podman/docker, registry access, DNS, NTP), try pulling and starting images manually, avoid mixing systemctl restarts with cephadm-managed containers, and use spec files as the source of truth.
AI-written summary · about
chapters (8)
- 0:00Introduction to Ceph ADM
- 2:17Troubleshooting challenges
- 6:55Methods for gathering logs
- 11:57Failed troubleshooting attempts
- 15:18Effective monitoring with tools
- 19:06Debugging best practices
- 22:21Automation and orchestration
- 25:49Questions and answers
related talks
16mTips and Tricks for Managing and Administering Ceph Clusters
45mContainers are hideously undebuggable black boxes and we never should have invented them
05mHealthier Ceph Clusters with Ceph-medic
40mJuggling Petabytes: Managing Ceph at Scale with Ceph-ansible
45mTroubleshooting containerized applications
10mCluster-scale Problem Solving
22mDeepSea: Deployment and Management of Ceph with Salt
34mGetting Started as a Rook-Ceph Developer