How FoundationDB powers SnowflakeDB's metadata!
summary
FoundationDB serves Snowflake as the central metadata store for 4+ years, holding ~200 entities (databases, tables, users, stats) and enabling features like cloning and time travel purely via metadata. It runs triple-replicated across cloud block storage with 33 clusters and ~1,000 processes, largest cluster sustaining about 300k reads and 100k writes/sec; small reads/writes hit ~sub-millisecond. An entity layer, watches, a Groovy CLI and SQL-facing read path are built on top; operational pain points include EBS/managed-disk IOPS limits, B-tree fragmentation and backup challenges, so monitoring, throttling and a dedicated FDB team are essential.
AI-written summary · about
chapters (13)
- 0:00Intro
- 0:40SNOWFLAKE: DATA WAREHOUSE BUILT FOR THE CLOUD
- 1:59SNOWFLAKE ARCHITECTURE: GLOBAL SERVICES
- 4:49MYSQL AS A METASTORE - NOT SIMPLE
- 8:22FDB FITS THE BUILDI
- 10:22ENTITY LAYER
- 12:46ENTITY SPECIFICATION
- 13:25ENTITY INTERFACE
- 14:46THE STORY OF FDB AT SNOWFLAKE
- 15:30METADATA FEATURES POWERED BY FDB
- 16:23TYPICAL PROD DEPLOYMENT
- 17:41OPERATING FDB IN THE CLOUD
- 20:42TAKEAWAYS
related talks
46mFoundationDB: The Best-Kept Secret of New Distributed Architectures!
11mNew Snowflake Capabilities and What They Mean for FoundationDB
16mFoundationDB and Node.js: Year in Production
34mFoundationDB, the Black Knight
18mFoundationDB Record Layer: Open Source Structured Storage on FoundationDB
25mBuild Anything with FoundationDB
24mHow We Saved 5x Migrating from Aurora to FoundationDB
20mUsing FoundationDB and the FDB Record Layer to Build CloudKit