Synchronous Reads, Asynchronous Writes
summary
Claim: keep reads synchronous and cacheable while sending writes through a message bus so services stay small and deployable. Implementation uses Synchronous reads via Sinatra and Jason over HTTP and Asynchronous writes through Rabbit mq fan‑out and topic exchanges, plus client gems, tyus/libcurl parallel fetches, and bunny or amqp for consumption. The deployed system processes about ten million messages a day on one server, tolerates queue failures by persisting state elsewhere, and accepts eventual consistency tradeoffs.
AI-written summary · about
chapters (22)
- 0:00Start
- 0:42Data Reads Through Services
- 1:44Monolithic Applications Don't Scale
- 2:41Services based Approach
- 3:53serialization format
- 4:53Data Store
- 5:38Joel Spolsky calls people that exhibit this behavior "Architecture Astronauts"
- 6:20Standard Rails Application
- 7:28Monolithic Applications Do Not Scale
- 7:47real world example
- 9:44engagement and post traffic is bursty
- 12:00multi-threaded and asynchronous parallelism
- 17:02package as gems
- 17:24versioning
- 17:54what about Beanstalk, Resque, Kestrel, or whatever?
- 18:14Event Based System
- 18:45Exchange Types
- 21:42error logging
- 22:45Bunny by Chris Duncan
- 23:53async write considerations
- 24:57Eric Brewer's CAP theorem
- 28:31single purpose services
related talks
18mRuby Messaging Patterns
41mDecouple all the things: Asynchronous messaging keeps it simple
33mAsynchronous Rails
43mRails services in the walled garden
56mScaling Rails API to Write-Heavy Traffic
36mIntroduction to Concurrency in Ruby
37mLet's build our own message streaming platform
39mAsynchronous and Non-Blocking IO with JRuby