Stop Hitting Yourself!
summary
Finds a 20‑year backup script that silently produces useless dumps because gzip in a pipe masks failures, leaving only about 12 days of valid backups instead of ~25,000 jobs; docker image drift breaks patching because the CI cached a 6.6 tag until manually updated so the fix is pull image before build; Kubernetes defaults cause outages — rollingUpdate + maxUnavailable=1 removes capacity during deploys, node drains take five minutes, and missing probes/requests let pods die under pressure, so add pod disruption budgets and readiness probes and manage manifests with Helm + Terraform and Vault for secrets.
AI-written summary · about
chapters (14)
- 0:00Start
- 1:12Stopping hitting yourself!
- 2:59Bash bashing
- 3:1620 years later...
- 7:20Changelog
- 9:44What actually happened
- 13:02Hello Docker build cache!
- 15:37Show me what happened!
- 17:00Update Strategy
- 17:13What would you expect?
- 19:25Kubernetes upgrade time!
- 20:06You forgot pod disruption budgets!
- 20:40Kubernetes node crashes
- 25:54You forgot probes!
related talks
37m10 Ways to Shoot Yourself in the Foot with Kubernetes, #9 Will Surprise You
36mKubernetes Confessions: Real Mistakes We’ve All Made (and What They Teach Us)
39m30 minutes of Failure
14mHow to Not-Mess-Up Production
32mSchroedingers Backup
35m10 Weird Ways to Blow Up Your Kubernetes
33mThe Gotchas of Zero-Downtime Traffic /w Kubernetes
41mAfterlife Tales