Real-Time Databases Explained: Why Meteor, RethinkDB, Parse & Firebase Don't Scale
summary
Claim: popular real-time databases struggle to scale because they either force every app server to see every write or they centralize update broadcast. Meteor uses oplog tailing or expensive polling, RethinkDB exposes changefeeds through proxies, Parse routes writes over Redis queues, Firebase exposes one big JSON with documented 1,000 client connections and limited query expressiveness. Built solution invalidDB runs on Apache Storm, implements a partitioned matching layer, achieves roughly 5 million match ops/sec per node and linear scaling across nodes and is in production.
AI-written summary · about
chapters (13)
- 0:00Start
- 1:55Push Based Data Access
- 5:14Real-Time Databases Meteor
- 16:22Firebase
- 20:59Evolving Collections
- 24:05Querying Expressiveness
- 30:04Filtering Stage
- 32:23Performance Numbers
- 33:21Query Api
- 35:05Limit and Offset Queries
- 37:02Questions
- 37:37What's Wrong with Your Database
- 42:21I Think It's It Doesn't Care Let's Take the First Question First so the Hardware Question Is Basically You Don't Need That Much Hardware To Make It Run so We Have a Development Server Where We Have a Local Storm Cluster and Everything Running Locally and You Can Work with that So in Order To Get Performance You You Know You Want To Use Something like some Server but It's Rather Low Overhead because Well It's all Natively Running in Java and Storm Is Pretty Efficient
related talks
40mBeyond Relational: Data storage for modern applications
08mWhen Worlds Collide: Frontend vs Database
35mAn Ode to 17 Databases in 39 minutes
29mWhen Web Scale Doesn't Make Sense
33mThe rise and fall of the full stack database
47mThe future of Real-time/offline/data
25mOffline-first data: Getting Bigger
29mSpeed up app development with prefabricated, extensible, open-source backends