How We Store and Analyze a Large Social Graph

on YouTube ↗44m

" summary

Describes storing and analyzing a large social graph (100M+ nodes, 1B+ edges) while new data are continuously added. Covers using graph databases DEX and Neo4j: fast bulk import techniques and caching strategies to speed traversals. Explains reasons for ultimately building a custom storage system tailored to the project's specific query and update workloads.

AI-written summary · about

" chapters (6)

1. 0:00Graph database introduction
2. 12:33Neo4j import challenges
3. 22:19Sparksee (TigerGraph) overview
4. 29:49Large graph testing
5. 36:43Conclusions and recommendations
6. 40:35Q&A

" related talks

• 54mDistributed Graph DBMSs — The Future for Analytics on Big Data?Pavel Velikhov · Saint HighLoad++ 2023
• 27mGraphOne: A Data Store for Real-time Analytics on Evolving GraphsPradeep Kumar · FAST '19
• 48mAnalytics on Large Graphs in S3: Tools, Approaches and Formats for OLTP and OLAPAleksey Teplov · Saint HighLoad++ 2025
• 50mBig Data on Smallish MachinesGuy Blelloch · Parallel and Distributed Algorithms for Inference and Optimization (Simons Institute, Oct. 21–24, 2013)
• 56mGraph Analytic SystemsZachary Hanif · Strata Data Conference 2018
• 32mLarge-Scale Graph Processing on Emerging Storage DevicesChangho Choi, Nima Elyasi, Anand Sivasubramaniam · FAST '19
• 21mSerious Network Analysis Using Hadoop & Neo4jFriso van Vollenhoven · Berlin Buzzwords 2012
• 30mHow Graph Data Science Can Turbocharge Your Knowledge GraphKristof Neys · Big Data LDN 2022