Towards Foundation Models for Graphs
summary
Argues that general foundation models for graphs largely do not yet exist but shows two concrete prototypes: ULTRA builds a graph of relations and enforces double permutation invariance to produce relative relation embeddings that enable inductive link prediction across previously unseen graphs and relations, matching or exceeding task-specific baselines after fine‑tuning. A second model combines closed‑form linear GNNs, permutation‑invariant attention and entropy normalization, trains on 120 nodes (Wisconsin) and outperforms standard GNNs trained on ~500,000 nodes across 31 datasets.
AI-written summary · about
chapters (15)
- 0:00Introduction to the workshop
- 1:33Defining foundation models
- 3:27Graph machine learning basics
- 6:21Core graph learning tasks
- 8:21Permutation invariance
- 11:13Mathematical graph representation
- 13:08Node classification techniques
- 17:55Graph neural networks
- 23:26Knowledge graph reasoning
- 31:29Designing graph foundation models
- 33:52Candidates for foundation models
- 37:18The Ultra model
- 42:20Foundation models for nodes
- 48:57Open challenges and takeaways
- 54:19Q&A and concluding remarks
related talks
01h 01mToward Universal Graph Representations: Foundations and Frontiers of Graph Foundation Models
01h 16mBoot camp on graph foundations models
18mEvaluating the effectiveness of Deep Learning Models for Foundational Program Analysis(…)
29mFactorized Graph Neural Networks (With Some Algebraic Cheating)
39mUsing Graph Neural Networks to Embrace the Dependency Within Your Data
31mGridFM - Towards Foundation Models for the Electric Grid
41mEnhancing LLMs with Graph Technology
16mPreview: Graph NNs