Data Prep for Graphs
summary
Shows concrete data‑prep steps: deduplication at scale with Datasketch and nearest‑neighbor tricks, numeric ID assignment with Keyvi/Redis, cycle detection and transitive‑closure fixes, and probabilistic quality scoring with PSL. Demonstrates an Arrow/Parquet representation and a kg‑lab style integration that claims orders‑of‑magnitude speedups for graph ETL and works end‑to‑end; cites Ray for distributed work and industry examples up to hundreds of trillions of edges.
AI-written summary · about
chapters (17)
- 0:00Introduction to graph data science
- 1:51Challenges in graph data science ecosystems
- 5:23Why data prep dominates graph projects
- 6:46Deduplication and entity resolution in graphs
- 7:49Unique identifiers and sparse matrix challenges
- 8:54Detecting and resolving cycles in graphs
- 10:23Transitive closure for graph inheritance
- 11:46Linking raw data into graph structures
- 12:49Denormalization pitfalls in enterprise graphs
- 13:51Link prediction and motif analysis
- 14:33Ontology alignment and taxonomy weaving
- 15:57Migrating relational data to graph models
- 16:41Data labeling for graph-based AI
- 18:48Visualization traps in graph analytics
- 20:10Data quality checks for graphs
- 21:54Parquet format for scalable graph data
- 23:19Resources and community for graph data science
related talks
01h 41mGraph Intelligence: Enhance Reasoning and Retrieval Using Graph Analytics
38mData Prep Kit: A Comprehensive Cloud-Native Toolkit for Scalable Da...
31mGraph Analytics - From the Whiteboard to Your Toolbox
40mPreparing data for LLM training with Data Prep Kit
30mAnalyze connections and find patterns in data with graphs
02mDBTB INT Alexis Roos r
53mTips and tricks for robust big data applications
26mGraph Analytics in Snowflake: Beyond Traditional SQL