Encrypting 6 Petabytes of Customer Data with AWS KMS
summary
Encrypting 6 Petabytes of Customer Data with AWS KMS describes a concrete rollout: use of AWS KMS with per-region keys, S3 object copy-encryption, SQS-driven fanout, DynamoDB status tracking and multi-part copies for files >10 MB. The system, called Kryptonite, hits KMS rate limits so it caches data keys, throttles with circuit breakers, and pauses work to avoid DB and KMS overload; initial migration takes months but completes, new uploads default to encryption, and a local S3 mock is open sourced.
AI-written summary · about
chapters (9)
- 0:00Start
- 6:01Looking back to 2015.
- 7:51Working with AWS KMS
- 10:34Watch Out! Rate Limits!
- 12:48What do we need?
- 18:11Encrypting a single 53 object by copying onto itself
- 19:29Resilience with Hystrix
- 20:30For Dev and Builds, use Docker images for everything
- 21:50Resting Encrypted-Status Today
related talks
26mProtecting your data in Amazon S3
51mUsing Key Management Service for data protection
45mLet's Encrypt Everything, Really Everything
50mImprove data control for customers with AWS KMS
58mUsing AWS data protection services for innovation and automation
45mAmazon S3 encryption and access control best practices
20mUnderstanding the shared responsibility model and AWS KMS XKS
20mApplication Secret Management with AWS KMS