Large-Scale Media Archive Migration to the Cloud
summary
A small team spends ~1.5 years moving 40 petabytes of studio assets into a cloud-native platform built on AWS, using Terraform/Nomad/Go workers and plain AWS CLI to push multi-part uploads tuned by chunk size, keeping developers out of S3 and enforcing chain of custody via a VFS overlay file system. Delivery runs about 5 petabytes per month over Aspera and EC2 nodes, migration timelines span months to a year, and a 6PB Isilon cluster is reduced to 300TB to cut cost and complexity.
AI-written summary · about
chapters (18)
- 0:00Intro
- 3:04A century of archive
- 3:54Digital intermediate
- 5:40Platform
- 7:51Use Cases
- 10:54Asset Identification
- 12:25Delivery
- 13:08Sample Package
- 13:36Orchestration
- 14:26Design
- 15:01ingest
- 15:25Centralized
- 16:17Bucket Policies
- 19:22Migration
- 21:01CLI Tools
- 22:23Supply Chain
- 25:22Storage Growth
- 26:52Takeaways
related talks
54mMigrating to Cloud Native with Microservices
28mCloud The Final Destination
25mAutobahn in the cloud
28mSaaS migration and modernisation
20mMigration of 300 microservices from AWS Cloud to
20mScaling Microservices and Distributed Databases in a Multi-Cloud World
37mDatabase Modernization at Cloud Scale: Minimizing Risk and Disruption
51mScaling Wix with microservices architecture