How Netflix Delivers Key-Value and Time-Series Storage at Any Scale
summary
Netflix runs Apache Cassandra at petabyte scale using client-side LZ4 compression, chunked writes (64 KB chunks, 2 MB pages) and dynamically bucket data into versioned chunk tables so no single node handles huge partitions. Idempotency tokens and page-based pagination keep latency bounded; tracing ingests 1–20M writes/sec and time-series stores tens of petabytes with namespace-based sharding and table lifecycles. Key-value serves 3,000 use cases across 400+ shards, yields operator and cost savings and claims up to 2x more storage versus TTL-based compaction.
AI-written summary · about
chapters (9)
- 0:00Netflix storage challenges
- 2:05Data abstraction layers
- 3:08Key concepts behind abstractions
- 14:45Key-value API design
- 20:52Time-series API design
- 23:39Impact and efficiency gains
- 29:34Real-world use cases
- 32:26Future innovations
- 33:09Q&A
related talks
46mCassandra Serving Netflix @ Scale
42mHow Netflix Built a Real-Time Distributed Graph for Internet Scale
44mTechniques Netflix Uses to Weather Significant Demand Shifts
20mNetflix Keystone - Streaming Data Pipeline @Scale in the Cloud
26mBuilding Financial Systems on Eventually Consistent DBs
30mScaling Time-Series Data to Infinity: A Kubernetes-Powered Solution with Envoy
52mHow Netflix Provisions Optimal Cloud Deployments of Cassandra
49mData Platform Architecture, Evolution, and Philosophy