Resiliency Patterns in Kubernetes
summary
Intuit runs a Modern SaaS platform that provisions a production-ready microservice in ~15 minutes and operates at 160 clusters, 6k nodes, ~5,400 namespaces, ~62k pods and 1,300 deployments/day. Common causes of 502/504 are unhandled SIGTERM and ALB keepalives on pod or node termination; solutions include configuring preStop hook + longer terminationGracePeriod, closing keepalive connections, and using ASG lifecycle hooks. An open source lifecycle-manager drains nodes and deregisters ALB targets and eliminates observed 504s.
AI-written summary · about
chapters (12)
- 0:00Start
- 1:30Intuit Products
- 1:49Goals for MSAAS
- 2:51Monolith to Microservices
- 6:50Intuit Development Platform - MSAAS
- 7:09Journey so far ... Services on MSAAS
- 9:01Request Flow
- 9:31Application Pod Termination
- 12:22Pod Spec
- 12:46Resiliency - Challenges with Infrastructure
- 14:21Keiko Lifecycle-Manager
- 15:39Conclusions
related talks
43mBuilding Resilient Services on Kubernetes
26mBuilding Resilient Services on Kubernetes
38mCaulking the deployment gap: absolutely zero downtime deployments in Kubernetes
27mRunning Kubernetes in Production A Million Ways to Crash Your Cluster
33mThe Gotchas of Zero-Downtime Traffic /w Kubernetes
45mKubernetes Resiliency
35mKubernetes, Lessons Learnt
29mKubernetes Workload Resiliency in Action: Beyond Basics