OK S3
summary
Odnoklassniki implements an S3-compatible service built on existing blob stores and Apache Cassandra for metadata, using SHA-256 content-addressed blocks with deduplication and optional versioning, a small finite-state machine per block to make block writes/deletes atomic across blob store and DB, and an off-heap block cache plus reactive parallel prefetch to speed reads. Wide Cassandra partitions and tombstones bite (one partition hit 7 GB), production handles peaks ~6000 RPS, buckets up to tens of TB, and per-instance network throughput tops ~2–3 Gbps.
AI-written summary · about
chapters (14)
- 0:00Introduction
- 1:46Why we needed S3
- 7:53S3 storage requirements
- 11:03Self-implementing S3
- 17:24S3 implementation architecture
- 27:42Data block storage and deduplication
- 30:10Object metadata storage
- 37:19Object listing methods
- 41:52Object lifecycle management
- 44:01Common S3 implementation mistakes
- 47:14Upload performance optimizations
- 51:50Download performance optimizations
- 58:35Testing results and production rollout
- 1:00:24Q&A
related talks
57mCassandra for Metadata Storage: Successes and Failures
54mOK S3: Building the System Ourselves
29mCassandra - an introduction
44mInside S3
53mApache Cassandra. Inception - all you need to know
23mHadoop & Cloud Storage: Object Store Integration in Production
27mSponsored Session: How is Amazon Keyspaces (for Apache Cassandra) Architected to Deliver Performance at Any Scale?
29mAccelerate your cloud architectures with the right storage options