Relational Deep Learning (RDL) enables predictive learning directly on relational databases by representing tables and foreign-key relationships as heterogeneous graphs. RelBench provides a benchmark for evaluating such models across databases with different schemas and downstream prediction tasks [1]. Heterogeneous Graph Neural Networks (GNNs), such as HGT [2], can model typed relationships, but their schema-dependent representations make transferring pretrained models across databases challenging. Recent studies have explored self-supervised graph pretraining [3], cross-domain graph pretraining [4, 5], and transferable graph reasoning [6]. However, how to identify and share compatible relations across different database schemas remains insufficiently explored. In particular, existing approaches do not systematically investigate how relation abstraction and importance-based selection affect information preservation, transferability, and negative transfer. This thesis aims to develop a cross-schema relation abstraction framework that characterizes, groups, and selectively preserves relations based on their structural properties. By constructing a shared relation representation space, the framework enables mixed GNN pretraining across multiple RelBench databases and investigates the following research questions: Research Questions: RQ1 — Relation Abstraction: How can structural and statistical characteristics of relations be used to construct a shared relation representation space across different database schemas? RQ2 — Cross-Schema Transferability: Can mixed pretraining across multiple databases improve downstream predictive performance on both seen and unseen schemas compared with random initialization and single-database pretraining? RQ3 — Abstraction and Selection: How do relation abstraction and importance-based selection individually and jointly affect information preservation, transferability, and negative transfer? RQ4 — Computational Efficiency: Can shared pretraining improve the trade-off between predictive performance and computational cost, including pretraining and downstream adaptation overhead?
Thesis Type |
|
Student |
Yixin Zhu |
Status |
Running |
Presentation room |
Seminar room I5 - 6202 |
Supervisor(s) |
Stefan Decker |
Advisor(s) |
Yixin Peng |
Contact |
peng@dbis.rwth-aachen.de |
Tasks
a) Dataset Preparation and Baseline Implementation
- Select at least three medium-scale RelBench databases with different schemas and prepare their heterogeneous graph representations.
- Implement a common GNN backbone and establish random-initialization and single-database pretraining baselines.
b) Cross-Schema Relation Abstraction and Selection
- Characterize relations using normalized structural statistics and local graph patterns, and develop strategies to map compatible relations into a shared representation space.
- Investigate relation importance-based selection to preserve informative relationships and reduce redundancy or negative transfer.
c) Mixed Pretraining and Cross-Schema Transfer
- Develop a shared GNN pretraining framework that integrates the abstracted relations from multiple RelBench databases.
- Evaluate transferability on downstream tasks involving both seen and unseen database schemas, including leave-one-database-out experiments.
d) Evaluation and Ablation Studies
- Compare downstream predictive performance and computational efficiency against randomly initialized and single-database pretrained models.
- Conduct ablation studies on relation abstraction, importance selection, and their combined effects, including abstraction granularity and negative transfer.
References
[1] Robinson, J., et al.: RelBench: A Benchmark for Deep Learning on Relational Databases. In: Advances in Neural Information Processing Systems 37 (NeurIPS 2024), Datasets and Benchmarks Track (2024). NeurIPS proceedings
[2] Hu, Z., Dong, Y., Wang, K., Sun, Y.: Heterogeneous Graph Transformer. In: Proceedings of The Web Conference 2020, pp. 2704–2710. ACM (2020). https://doi.org/10.1145/3366423.3380027
[3] Hou, Z., et al.: GraphMAE: Self-Supervised Masked Graph Autoencoders. In: Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD 2022), pp. 594–604. ACM (2022). https://doi.org/10.1145/3534678.3539321
[4] Zhao, H., Chen, A., Sun, X., Cheng, H., Li, J.: All in One and One for All: A Simple yet Effective Method towards Cross-domain Graph Pretraining. In: Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD 2024), pp. 4443–4454. ACM (2024). https://doi.org/10.1145/3637528.3671913
[5] Liu, H., et al.: One For All: Towards Training One Graph Model For All Classification Tasks. In: International Conference on Learning Representations (ICLR 2024) (2024). ICLR proceedings
[6] Galkin, M., Yuan, X., Mostafa, H., Tang, J., Zhu, Z.: Towards Foundation Models for Knowledge Graph Reasoning. In: International Conference on Learning Representations (ICLR 2024) (2024). ICLR proceedings
- Solid foundation in Machine Learning / Deep Learning and graph representation learning.
- Good programming skills in Python.
- Experience with PyTorch and graph-learning frameworks such as PyTorch Geometric or DGL.
- Familiarity with Transformers, attention mechanisms, and self-supervised learning.
- Basic knowledge of Knowledge Graphs, heterogeneous graphs, and temporal graph learning is beneficial.
- Experience with NLP models or text embeddings is helpful for representing textual attributes of academic entities.