Understanding Databases for Distributed Docker Applications
summary
An open source distributed DB built on Lucene and Elasticsearch with a custom SQL planner uses Docker plus Weave to spin clusters across hosts; a live demo shows an 11‑node cluster holding 666 million records and ~50 GB of time‑series data for a step tracker, queries returning in as little as 0.012s and typical reads ~0.18s. Nodes are peer‑equal by default, replication and rebalancing work when nodes are added or killed, and consistency is eventual not immediate, requiring design tradeoffs.
AI-written summary · about
chapters (19)
- 0:00Introduction to talk
- 0:52Why distributed databases
- 3:04Evolution of scaling applications
- 6:41Challenge of persistent data in Docker
- 7:43What is CrateDB
- 9:06CrateDB architecture overview
- 11:19CrateDB with Docker basics
- 13:55Demo: CrateDB admin console
- 15:42Demo: Large distributed cluster
- 18:16Demo: Scaling and rebalancing
- 21:35Demo: Handling node failures
- 23:03Client libraries for CrateDB
- 25:09Eventual consistency explained
- 27:27Coding examples with CrateDB
- 29:11Docker Compose for CrateDB
- 29:54Docker Machine and Swarm
- 32:04Solution: Weave for networking
- 34:15Use cases and limitations
- 35:19Summary and Q&A
related talks
47mRunning Databases in Docker at Scale
45mDealing with TBytes of Data in Realtime
58mIntroducing ScaleDB, the New Clustered, Distributed Storage Engine for MySQL (Part 1)
29mModern Database Architecture: From SQL to NoSQL & Distributed SQL
22mRunning High Performance And Fault Tolerant ElasticSearch ...
34mPanel Discussion: Databases
38mDocker and Databases
48mContainer: check! Aber wohin mit Persistent Data, Big Data oder Fast Data?!