Optimizing Video Storage Via Semantic Replication
summary
Facebook treats different encodings of the same video as semantic replicas and compacts cold videos by keeping one physical copy per encoding instead of two, cutting storage for those assets by about 50%. Videos are chunked; initial placement policy volumes caused metadata IO to jump from ~4 reads to over 100, so the system introduces petabyte-scale mega volumes and cross-region tuples called semantic volumes to keep encodings in distinct failure domains and to allow changing per-encoding replica counts when videos go hot again.
AI-written summary · about
chapters (10)
- 0:00Video storage optimization intro
- 1:04Semantic replication concept
- 3:55Physical vs semantic replicas
- 6:02Temporal access patterns
- 7:06Storage system requirements
- 8:28Building the replication engine
- 9:49Blob storage overview
- 11:15First replication attempt issues
- 14:02Mega vs semantic volumes design
- 16:30Trade-offs and future steps
related talks
20mBudgeting Bytes: Acing Cost-Efficient Video Storage
33mArchitecting Scalable Multi-Modal Video Search
35mLarge-Scale Video Intelligence
17mGemino: Practical and Robust Neural Compression for Video Conferencing
36mWhen Semantic Search Breaks in your RAG systems: RAM Walls, Silent Failures, and the Architecture Decisions
14mRegion-based Content Enhancement for Efficient Video Analytics at the Edge
26mPetabyte-Scale Video Curation on CoreWeave
21mAI Storage Blueprint