How Shopify Sharded Rails

on YouTube ↗23m

" summary

Shopify shards a once-single Rails DB in 2013 to handle ~150k requests per minute with peaks near 400k and $1.6B GMV and ~88k paying shops, doubled the number of databases and builds a block-based connection switcher plus a charted-model concern. Primary keys use a MySQL auto-increment trick instead of a separate ID service. Verifiers catch cross-chart queries, a locking migrator copies shops live, and production sees 80ms average response time through CyberMonday.

AI-written summary · about

" chapters (10)

1. 0:00Start
2. 2:20Cons of sharding
3. 6:09Sharding Shopify
4. 8:34Denormalize
5. 9:33Primary keys
6. 11:59Connection switching
7. 15:02Rebalancing
8. 17:57What about JOINS
9. 19:33How long did it take?
10. 20:58OMG Cybermonday

" related talks

• 41mHow Shopify Scales RailsJohn Duff · Big Ruby 2013
• 40mHow Shopify Scales RailsJohn Duff · RailsConf 2013
• 37mScaling Rails for Black Friday and Cyber MondayChristian Joudrey · RailsConf 2015
• 17mHow We Deploy ShopifyKat Drobnjakovic · RailsConf 2016
• 31mA Rails performance guidebook: from 0 to 1B requests/dayCristian Planas · RubyConf TH 2022
• 56mScaling Rails API to Write-Heavy TrafficTakumasa Ochi · RailsConf 2021
• 32mRails Security at ScaleJack McCracken · RailsConf 2019
• 29mMigrating Shopify’s Core Rails Monolith to TrilogyAdrianna Chang · Rails World 2023