Highly Scalable Image Storage with AWS Serverless
summary
Replaces ALB/EC2/EFS with S3 for object storage, API Gateway + Lambda for upload/download, S3 events for postprocessing and SNS/SQS fan‑out, and DynamoDB for metadata (storing large project blobs in S3). Processes millions of images daily, migrates customer-by-customer asynchronously so users stay live, uses S3 presigned URLs for uploads and DynamoDB single-table patterns, and runs into Lambda/API Gateway concurrency quotas and tracing/observability challenges that require architectural workarounds.
AI-written summary · about
chapters (11)
- 0:00Intro
- 2:23Agenda
- 3:02Application architecture in 2017
- 8:56Application architecture in 2018 & 2019
- 10:52Motivation behind the reimplementation
- 16:43Current architecture
- 27:08Challenges & lessons learned
- 27:331. Architect with AWS serverless quotas in mind
- 38:272. General architectural decisions
- 41:273. Challenging to trace all requests
- 45:36Outro
related talks
50mDeveloping Highly Scalable Image Storage Solution with AWS Serverless
21mThe best compute for your storage - Amazon S3 & AWS Lambda
49mServerless Architectural Patterns and Best Practices
54mServerless Architectures: no servers allowed!
22mS3 & Lambda - flexible pattern at the core of serverless applications
51mArchitecting for Hyperscale with AWS: Navigating Real-world Serverless Challenges
31mAccelerate business innovation with serverless applications
41mUnlocking Faster & Efficient Data Processing w/ Serverless