Taking Storage for a Ride
summary
Built a global operational store that moves trips out of a single Postgres into a sharded, MySQL-backed system that scales to 300+ datastores and handles ~5M trips/week (2B total trips milestone). Core pieces: append-only, schemaless rows keyed by 128-bit UID sharded (mod 4k), sharded MySQL clusters with hinted-handoff buffer writes, secondary indices sharded by index key (eventual consistency), a partition-read tail API for trigger-driven processing, and gold-state self-healing agents for automated splits, restores and failover.
AI-written summary · about
chapters (5)
related talks
42mDatabases unveiled; choose wisely
54mBreakout Session
36mPostgres at any Scale
32mData @Scale Seattle- Eric Hwang introduced by Ganapathy Krishnamoorthy
43mJust Use Postgres.
01h 04mHow do you store your data?
37mScaling Vinted: A System That Never Stands Still
39mZero-Downtime Rebalancing and Data Migration of a Mature Multi-Shard Platform