Deep Stream Dynamic Graph Analytics with Grapharis
summary
Grapharis implements a real-time deep graph learning pipeline using Flink + TensorFlow pipeline that trains and serves embeddings online without full retraining. It stores per-vertex state, broadcasts model updates, and uses prioritized sampling by per-vertex loss so new or hard vertices get more training. Inference avoids redundant work with session-window batching of matrix multiplies; experiments on PubMed and Reddit show faster recovery when a new class appears (step 16) and throughput rises with workers and window size at the cost of higher end-to-end delay.
AI-written summary · about
chapters (20)
- 0:00Start
- 0:13Machine learning on graphs
- 3:03Pipeline
- 3:27Dynamic dataset
- 5:56Duplicate computations
- 6:24Graph updates
- 7:13Problems with batch approaches
- 7:40State
- 8:02Message passing - addition
- 9:13Iterations
- 10:14Neural network execution
- 10:39Execution plan
- 11:01Dynamic model
- 12:00Solving Iteration Deadlocks
- 12:37Datasets
- 13:09Baselines
- 13:28F1 score
- 14:28Training time
- 14:44Scalability - Throughput
- 15:12Delay - Pubmed
related talks
33mMonitoring a TV streaming service with AI - from PageRank to graph convolutions
25mReal-Time Fraud Detection at Scale
26mDeep Graph Library: Deep Graph learning at scale
13mMarius: Learning Massive Graph Embeddings on a Single Machine
43mEmpowering Real-Time ML Inference and Training with GRIS
20mUsing Flink for Versatile Feature Engineering Pipelines
43mBuilding Billion Node Graphs for Machine Learning
41mAdding Generative AI to Real Time Streaming Pipelines