Distributed Java DBs Under the Hood: Main Components and Interactions Between Them
summary
Uses Apache Ignite as a model and implements a simple Java distributed cache with string key/value put/get API, in-memory storage as a concurrent hashmap, networking over sockets, and etcd for cluster discovery. Demonstrates partitioning with 10 partitions, a mapper that hashes key→partition→node, request routing, and rebalancing; shows a basic performance goal of linear scale (50k TPS per node → 150k on 3 nodes). Uses random hashing to ensure minimal disruption on node joins; reference implementation is on GitHub and runs end-to-end.
AI-written summary · about
chapters (10)
- 0:00Introduction
- 1:55Requirements for distributed DB
- 6:49Basic client-server architecture
- 9:19Adding cluster awareness
- 13:54Request routing mechanism
- 17:03Data rebalancing process
- 20:55Data partitioning technique
- 24:22Minimal disruption challenge
- 33:50Final architecture summary
- 36:17Q&A session
related talks
01h 08mDistributed Java Databases Under the Hood
33mDistributed Database DevOps Dilemmas? Kubernetes to the rescue!
59mDistributed applications using Hazelcast
43mCaching and Beyond - Smarter Data Processing with Java
32mDistributed Database DevOps Dilemmas? Kubernetes to the Rescue
30mDistributed Computing with Hazelcast
53mLeveraging Caching For Cloud Native Applications
47mHunting issues in a Java app with 31G heap and 500G off-heap: an Apache Ignite case study