Database Sharding the Right Way Easy, Reliable, and Open source
summary
A production-ready open-source RDBMS called Kubrick claims native sharding with built-in high availability and load balancing, shipped as simple installs plus three config files (shard config, connection list, custom shard library). Deployment uses a Medley middleware (broker/proxy/reader), 64-bit unique ID generation, and heterogeneous shards (Qbrad, MySQL, planned Oracle). Internal use spans 118 services and a largest deployment in Japan; benchmarks show ~14k req/s (up to 50k tps claimed) and large latency and load drops with sharding. Known limits: ~90% MySQL compatibility, no cross-shard aggregation yet, and relatively high memory needs.
AI-written summary · about
chapters (17)
- 0:00Introduction and speaker intro
- 2:56What is Kubrick database
- 8:01Why sharding matters now
- 10:54Kubrick’s native sharding features
- 16:33Setting up sharding in Kubrick
- 20:58Performance benchmarks and results
- 26:28Advantages of Kubrick sharding
- 31:11Q&A session begins
- 31:35Mac OS support limitations
- 33:04Master database as single point of failure
- 36:27Multi-continent sharding architecture
- 38:19Master-slave replication in sharding
- 39:23MySQL compatibility gaps
- 41:38Limitations and resource requirements
- 43:43Aggregation across shards
- 44:51Programming language support
- 46:00Closing remarks and Q&A wrap-up
related talks
27mAutomatically shard & scale-out traditional databases on K8s
23mSharding for hyperscale and geographically distributed relational databases
32mSharding for mission-critical, internet-scale apps
52mTo Shard or Not to Shard
42mPostgreSQL Distributed & Secure Database Ecosystem Building
52mDatabase sharding at the load balancer level
25mHow Sharekhan Scaled Trading Platform with Yugabyte's Distributed SQL
29mEnterprise Scale: How a Distributed Architecture Can Deliver Scale and Resilience