DBIS

Background and Motivation

Large-scale knowledge graphs (KGs) such as Wikidata encode both rich relational structure and natural-language descriptions of entities and relations. Despite their scale and coverage, most KG models are trained narrowly for a single objective, e.g. link prediction, and generalize poorly across tasks. In parallel, the foundation model paradigm has reshaped language and vision: a single model is pretrained on a broad signal and then adapted, with little task-specific supervision, to a wide range of downstream problems. Transferring this paradigm to knowledge graphs is non-trivial. KGs mix hierarchical, cyclic, and flat relational patterns within one graph; entities carry no natural ordering; node attributes are continuous while edge types are discrete; multiple relations may hold between the same pair of entities; and real-world graphs reach hundreds of millions of nodes. A foundation model for KGs must therefore reconcile heterogeneous geometry, jointly model continuous and discrete signals, integrate textual semantics, and remain scalable, all while staying general enough to support diverse downstream tasks through lightweight adaptation.

Research Objective

The thesis investigates how a generative graph foundation model can be pretrained on large-scale, text-rich knowledge graphs and subsequently adapted to multiple downstream tasks, including knowledge graph completion, text-conditional subgraph generation, and graph anomaly detection. The central question is how a single pretrained model can capture the structural and semantic regularities of a knowledge graph well enough to transfer across tasks with minimal task-specific tuning.

Research Questions

1. Representation. How can entities and relations with heterogeneous relational geometry and accompanying text be embedded in a unified representation space that preserves both structural and semantic regularities?
2. Pretraining. What self-supervised generative pretraining signal on knowledge subgraphs yields representations that transfer broadly across downstream tasks?
3. Adaptation. How can a single pretrained model be efficiently fine-tuned or prompted to serve structurally different downstream tasks under a unified interface?
4. Scalability. How do representation quality and transfer behaviour scale with model size, data size, and graph scale?

Scope and Technical Approach

The work combines several techniques into a coherent pretraining-and-adaptation pipeline. These include geometry-aware representation learning over heterogeneous (curved and flat) embedding spaces; permutation-equivariant graph transformer architectures; text-conditioned encoding for semantic grounding; multi-task self-supervised objectives; and generative modelling of graphs via flow matching, which serves as the generative backbone for joint continuous–discrete dynamics. Rather than centring the thesis on any single one of these techniques, the contribution lies in their integration into a transferable foundation model and in the empirical study of its pretraining and downstream behaviour.

Expected Contributions

• A unified formulation of knowledge-graph foundation-model pretraining that jointly accounts for relational geometry, textual semantics, and the continuous–discrete nature of graph data.
• A pretrained model that transfers to several downstream KG tasks through a shared adaptation interface.
• An empirical analysis of transfer, ablations over the contributing techniques, and scaling behaviour across model, data, and graph size.

Evaluation

Pretraining is conducted on a large text-rich KG (e.g. Wikidata-scale data), with downstream evaluation on standard benchmarks for knowledge graph completion and related tasks. Reported results follow reproducible practice (fixed seeds, logged configurations, mean ± std over multiple runs).