Shopify's Move from the Data Centre to the Cloud
summary
Shopify migrates over 600,000 shops and ~80,000 rps from colo to cloud by sharding shops into 100+ pods, moving shops one-at-a-time with a shop mover that uses the open source Ghostferry to do online MySQL copies and binlog replay for near-zero downtime. Kubernetes runs thousands of nodes and dense pods, forcing a split of stateful and stateless clusters, solved with IP aliasing and an ILB-based cross-cluster DNS trick. Operational tooling includes active failover and a safer kubernetes-deploy to avoid blast-radius mistakes.
AI-written summary · about
chapters (11)
- 0:00Start
- 3:47Shopify early 2013
- 4:50Sharding - 2013
- 5:56Multi DC - 2015
- 20:03Pod Routing 2.0
- 20:59Shop Mover
- 29:53Shopify Core k8s vi
- 31:53k8s Scaling Problems
- 32:48k8s Scaling Solutions
- 34:38Shopify Core k8s v2
- 36:11kubernetes-deploy
related talks
33mBuilding Shopify's PaaS on Kubernetes
46mKeep Building Fresh: Shopify's Journey to Kubernetes
19mBuilding an On-Premise Kubernetes...
20mMigrating a Large Scale Search Dataset in Production in a Highly Available...
04mKeynote: One Deploy Plane, Every Runtime: How Shopify Generalized Cloud Native Beyon...
32mScaling Shopify's Search: Enhancing Elasticsearch Resilience With Kubernetes and KE...
32mOperating Rails in Kubernetes
40mSecuring Shopify's PaaS on GKE [I]