An approach to unite tables and persistent queues in one system
summary
YDB unites row/column tables and persistent queues called topics so you can read from a topic, enrich with table data and write back atomically: ACID transactions across tables and topics. Topics implement Kafka-compatible APIs and exactly-once/at-least-once delivery; internal clusters sustain up to 200 GB/s write throughput. Transactions use a Calvin-style ordering and split non-deterministic work into deterministic steps; tests with 100 partitions, 100 writers and 100 MB/s show minimal latency for ~10 KB messages but higher cost for large messages.
AI-written summary · about
chapters (7)
- 0:00Introduction
- 1:11YDB architecture overview
- 4:24Uniting tables and topics
- 9:44Technical aspects of transactions
- 13:26Transaction execution process
- 27:08Performance impact and conclusions
- 29:43Q&A and closing
related talks
01h 00mParallel Asynchronous Replication b/t YDB Database Instances
46mYDB Topic Service: How We Improved Message Queue Performance
38mTransactional Work with Topics
51mComparing the Scalability of Kafka and YDB Topics
40mData bus features for very large installations: the YDB Topics example
47mState machines for product data from YDB and queues
27mIntroduction to YugabyteDB – Design and Architecture
44mYDB Java SDK: The Battle for Performance