Zen: Pinterest's Graph Storage Service
summary
Zen is a graph storage service Pinterest built on HBase that exposes nodes, edges and CRUD plus query APIs, powering home feed, messages and the interest graph. It implements unique and non-unique property indexes, materialized edge counts, edge-score pagination, soft-deletes, namespaces and optional reverse edges. At production scale it runs hundreds of thousands of requests/sec and over 5M HBase ops/sec; smart feed required handling about 1M edge insertions/sec, solved by client/server batching to avoid HLog bottlenecks. Consistency favors performance with offline repair jobs; open sourcing is planned.
AI-written summary · about
chapters (9)
- 0:00Introduction to Zen
- 0:25Motivation for Graph Data
- 2:58Zen API and Architecture
- 5:51Example and Features
- 9:41Zen Adoption and Usage
- 10:41Implementation Details
- 14:13Performance Challenges
- 17:34Data Consistency and Future
- 20:19Q&A Session
related talks
24mPinterest’s Exploration of Distributed SQL
01h 03mZenoh: The Edge Computing Data Fabric
29mImplementing a Graph Database based on PostSQL
25mHundreds of Terabytes of Data
25mThinking in Graphs
44mUSER-FACING, REALTIME ANALYTICS USING APACHE PINOT
36mEclipse Zenoh
20mZeno: Diagnosing Performance Problems with Temporal Provenance