Build distributed persistent queue on FoundationDB
summary
Tigris uses FoundationDB to implement a persistent queue on FoundationDB that atomically writes metadata and queued events in the same transaction across 11 regions, so no events are lost. Work is presharded and schedulers map one to one to shards, shard assignment runs over a gossip protocol, workers take sharded leases and renew them, and priorities map to separate deployments. Operationally it replaces a separate messaging system and achieves multi‑region replication in seconds per the shown graphs.
AI-written summary · about
chapters (9)
- 0:00Introduction and Speaker Background
- 1:32Overview of Tigress Data
- 2:43Designing a Global Object Store
- 5:18Foundation DB: The Core of Our Metadata Storage
- 6:38Building the Queuing System
- 10:39Work Distribution and Shard Assignment
- 12:52Processing Queue Items
- 14:43Recap and Performance Insights
- 18:53Q&A Session
related talks
37mTechnical Overview of FoundationDB
22mSolving Everyday Data Problems with FoundationDB
25mBuild Anything with FoundationDB
53mBuilding Distributed, Multi-Tenant Databases with FoundationDB and Rust
24mHow We Saved 5x Migrating from Aurora to FoundationDB
34mFoundationDB, the Black Knight
16mFoundationDB and Node.js: Year in Production
11mConsistent Caching in FoundationDB