Herding Elephants
summary
Claim: Heroku Postgres runs the largest fleet of Postgres databases using a handful of Ruby apps. Concretely this is a constellation of Sinatra apps, hundreds of background workers (50 distinct process types, many with 200+ instances), and thousands of databases checked at least once a minute. Monitoring is done outside‑in via an append‑only observations table, a ring queue of stateless workers, codified playbook resolvers (restart EBS, failover), and state machines for incidents; simple restarts fix many faults, unresolved cases escalate to humans.
AI-written summary · about
chapters (4)
related talks
13mBuilding reliable infrastructure with Postgres
01h 03mRuby at Scale
11mGiving your Heroku App highly-available PostgreSQL
22mLessons from Automatic Incident Resolution for a Million Databases
43mPolyglot Paralellism: A Case Study in Using Erlang and Ruby at Rackspace
40mHacking the Pachyderm: Scaling Servers and People
26mBiggish Data With Rails and Postgresql
34mIndustrial Rails