Pinterest's Journey from VMs to Containers [I]
summary
Pinterest runs on tens of thousands of EC2 hosts serving ~200M MAUs and hundreds of PB of storage and moves from VM images to Docker images, achieving 75% of stateful services in Docker by June 2017. An eight-week POC compares Kubernetes, Mesos/Marathon, Aurora and ECS and chooses Kubernetes while accepting gaps for big data; engineering builds a service description language, a custom CNI proxy daemon that allocates AWS ENIs, IAM proxying via "drum", parallel image prefetching, and begins productionizing k8s and Spark support.
AI-written summary · about
chapters (16)
- 0:00Introduction
- 1:03How Pinterest was born
- 6:41Vision for the Compute Platform
- 14:32Introducing Lida
- 16:41Service Description Language
- 18:07Under the hood
- 19:35Migration
- 20:02Learnings
- 22:18Overview
- 22:31Evaluation
- 22:41POC
- 24:33Networking
- 25:47Limit Data
- 26:42Network Setup
- 28:27IAM Setup
- 29:34Future Roadmap
related talks
01h 02mContainers in Production: It’s Like Orchestrating Cats
12mHow we deploy Spinnaker microservice pods via Spinnaker at Pinterest
33mGovernance on K8s: How to Solve Ownership, Metering & Capacity Planning
46mAround & After Kubernetes: The Principles and Ideas that Guide Us
05mBare Metal to Containers to K8s: How We Got Here
44mFrom single container to applications at scale
01h 06mSo many Docker platforms, so little time...
43mContainers Change Everything