Zero Trust at Shopify Scale: Automating MTLS Across Thousands of Services
Talkat KubeCon + CloudNativeCon Europe 2025 · on Friday, 2025-04-04· at 11:00· Level 1 | Hall Entrance N10 | Room H·
summary
Shopify automates mTLS for thousands of services across millions of pods, thousands of nodes, and hundreds of clusters using Google Cloud Certificate Authority Service plus a mix of cert-manager and a custom Go certificate-loader that issues via workload identity, stores certs in Kubernetes secrets, and renews with deterministic, randomized cronjobs (four times daily). They built a custom Kafka principal parser to map SPIFFE SAN URIs, used Argo CD sync waves for bootstrapping, push metrics via statsd/Pushgateway to Prometheus, and the demo shows successful rollovers and alerting for a permissions-induced issuance failure.
AI-written summary · about
chapters (20)
- 0:00Shopify’s MTLS journey
- 1:04Why zero trust matters
- 3:08Workload identity and passports
- 6:00Spiffy IDs at Shopify
- 7:47Kafka challenges with Spiffy IDs
- 8:28Service-to-service auth options
- 9:34Avoiding service mesh overhead
- 11:00Ingress-nginx MTLS setup
- 13:24Risks of unrotated credentials
- 16:14Certificate management at scale
- 17:17CA rotation planning
- 18:42Cert-manager for Kubernetes
- 19:23Certificate management beyond K8s
- 21:32Unifying cert tools
- 22:33Demo setup with Argo CD
- 24:43Cert renewal and alerting
- 25:24Observability for cert failures
- 27:28Monitoring job metrics
- 28:54Full demo walkthrough
- 34:29Key takeaways
related talks
24mOur Journey of Implementing TLS at Scale for Services on Kubernetes
31mPractical TLS Advice for Large Infrastructure
55mZero Trust Security For AlwaysOn APIs
40mSecuring Shopify's PaaS on GKE [I]
33mBuilding a Secure, Multi-Protocol and Multi-Tenant Cluster for Internet-Facing Services [A]
32mAchieving Mutual TLS: Secure Pod-to-Pod Communication Without the Hassle
29mZero Trust Security with Service Mesh
38mImproving Secure Pod-to-Pod Communication Within Kubernetes Using...