Ø Kubernetes Failover Improvement: Non-Graceful Node Shutdown
summary
Kubernetes had a problem where StatefulSet pods with attached persistent volumes would remain Terminating when a node became unresponsive because the control plane waited 300s. The talk describes the Non-graceful node shutdown feature (alpha in v1.24, GA in v1.28) which adds a node.kubernetes.io/out-of-service=no_shutdown:NoExecute taint so controllers forcibly evict pods and detach PVs; combined with Node Health Check Operator and Self Node Remediation the demo shows pods get moved and PVs reattach (observed ~6-minute downtime).
AI-written summary · about
chapters (9)
- 0:00Kubernetes failover issues
- 1:59Kubernetes failover behavior
- 5:42Persistent volume failover
- 8:15Non-Graceful Node Shutdown
- 10:49Demo: Non-Graceful Node Shutdown
- 15:22Non-Graceful Node Shutdown in production
- 18:13Demo: Node health check
- 22:23Conclusion
- 23:51Q&A
related talks
37mHow To Handle Node Shutdown In Kubernetes
19mGraceful Node Shutdown – A Beginner’s Journey of Kubernetes Development
33mNot so graceful: K8s node failure and stateful workloads
42mImproving Availability for Stateful Applications in Kubernetes
29mStateful Workloads in Kubernetes: A Deep Dive
28mThe Magic of Kubernetes Self-Healing Capabilities
38mCaulking the deployment gap: absolutely zero downtime deployments in Kubernetes
31mBeyond the Start Button: The Hidden Life of Pod Termination in Kubernetes