Read and write considered harmful
summary
Claim: reads and writes behave radically differently and treating them the same damages architecture. Evidence: shared writes show zero scalability on contention, a CTO faces 5x/year growth hitting one central DB, MySQL replication and sharding buy only months, and a test with 17,000 duplicate keys destroyed hashing performance. Patterns and tools: use Lambda architecture, Kafka, CDNs, memcache and CQRS to separate write-heavy ingestion from read-serving. Built system with tens of thousands TPS decouples logging and survives load tests — it works.
AI-written summary · about
chapters (22)
- 0:00Intro
- 2:27Reads
- 6:35Dependencies
- 7:39REST APIs and rules
- 9:11REST APIs and schemas
- 10:59REST APIs and processes
- 12:35Typical system scaling path
- 15:26Scaling problems • Partitioning or sharing works to an extent
- 17:47Avoid sharing mutable data
- 21:42questions about data access
- 24:39Non-primary key access - Finding items by value, not by key
- 25:54Range scans and sequential access - Requires ordering Leoperators, ordering costs • Requires iberators/cursors/traversal state
- 28:45Read/write ratio
- 29:52Working set size and skew • How much of the common data will fit in main
- 32:38Consistency • Do all copies of the data need to be exactly up- to-date right now
- 44:15Reader/writer vs data flow
- 45:00Sync vs async systems
- 47:27Data flow and sync/async
- 49:41Content management example
- 50:50Larger example
- 53:55Command Query Representation Separation (CORS)
- 57:03CORS examples
related talks
57mRead and write considered harmful
40mArchitecting for Data-Intensive Workloads
48mDesign of data-intensive applications
30mDatastore Axes: Choosing the Scalability Direction You Need
52mTo Shard or Not to Shard
29mApplying (D)DDD and CQ(R)S to Cloud Architectures
01h 19mKeynote: I Can Write the Code But Getting Something Done Is Another Matter
49mRethinking Reactive Architectures