Read and write considered harmful
summary
Claim: reads and writes behave fundamentally differently — reads scale horizontally, while writes cause fan‑in contention and break caching. Gives concrete patterns: memcache/Redis for reads, sharding for write keys, Kafka/Kinesis for durable replay, S3 + batch (Hadoop/Athena) for immutable slow path, and CQRS/structure changing to store for write and transform for read. Cites Twitter fan‑out, a warehouse deadlock worsened by synchronous updates, and an hourly batch that removed complexity — practical fixes that work.
AI-written summary · about
chapters (13)
- 0:00Start
- 1:11Overview
- 12:52REST APIs and schemas
- 16:07Typical system scaling path
- 21:44Avoid sharing mutable data
- 25:56Full vs incremental
- 29:28Reader/writer vs data flow
- 35:29Sync vs async systems
- 37:42Content management example
- 39:46Larger example
- 42:13Command Query Representation Separation (CORS)
- 50:22Immutability
- 54:21Summary
related talks
01h 14mRead and write considered harmful
48mDesign of data-intensive applications
50mLet's talk about the cache!
30mDatastore Axes: Choosing the Scalability Direction You Need
40mArchitecting for Data-Intensive Workloads
18mAvoiding Conflicts in Highly-Concurrent Applications
49mRethinking Reactive Architectures
01h 03mArchitecting a large social network without breaking the bank