Kategorie: ‘Theses’
Cross-Schema Transferable Graph Learning on Relational Databases (Master Thesis)
Relational Deep Learning (RDL) enables predictive learning directly on relational databases by representing tables and foreign-key relationships as heterogeneous graphs. RelBench provides a benchmark for evaluating such models across databases with different schemas and downstream prediction tasks [1]. Heterogeneous Graph Neural Networks (GNNs), such as HGT [2], can model typed relationships, but their schema-dependent representations make transferring pretrained models across databases challenging.
Recent studies have explored self-supervised graph pretraining [3], cross-domain graph pretraining [4, 5], and transferable graph reasoning [6]. However, how to identify and share compatible relations across different database schemas remains insufficiently explored. In particular, existing approaches do not systematically investigate how relation abstraction and importance-based selection affect information preservation, transferability, and negative transfer.
This thesis aims to develop a cross-schema relation abstraction framework that characterizes, groups, and selectively preserves relations based on their structural properties. By constructing a shared relation representation space, the framework enables mixed GNN pretraining across multiple RelBench databases and investigates the following research questions:
Research Questions:
RQ1 — Relation Abstraction: How can structural and statistical characteristics of relations be used to construct a shared relation representation space across different database schemas?
RQ2 — Cross-Schema Transferability: Can mixed pretraining across multiple databases improve downstream predictive performance on both seen and unseen schemas compared with random initialization and single-database pretraining?
RQ3 — Abstraction and Selection: How do relation abstraction and importance-based selection individually and jointly affect information preservation, transferability, and negative transfer?
RQ4 — Computational Efficiency: Can shared pretraining improve the trade-off between predictive performance and computational cost, including pretraining and downstream adaptation overhead?
Budget-Aware Language–Graph Pretraining (Master Thesis)
Knowledge-augmented question answering (QA) combines language understanding with relational evidence from knowledge graphs (KGs). QA-GNN [1] and GreaseLM [2] model language–graph interactions, while DRAGON [3] learns joint representations through masked language modeling and KG link prediction.
However, retrieved evidence can exceed practical text and graph budgets. Fixed-size inputs and sampling or truncation may discard useful relational paths, retain redundant evidence, and overlook the cost of retrieval and selection. It also remains unclear whether budget-aware evidence selection should be applied only downstream or incorporated into joint pretraining. This thesis investigates structure-preserving selection during pretraining and its impact on the end-to-end accuracy–cost trade-off.
Research Questions
RQ1 — Evidence Selection under Budget Constraints. How can relevance, redundancy, complementarity, and structural integrity be jointly considered when selecting knowledge within text and graph budgets? Which selection principles retain decision-relevant evidence more effectively than random sampling, sequential truncation, or relevance-based ranking?
RQ2 — Selection and Joint Pretraining. Does incorporating budget-aware selection into language–graph pretraining outperform compressing inputs only at the downstream stage? How do self-supervised objectives, cross-modal interaction, and training-budget settings influence the outcome?
RQ3 — Accuracy and End-to-End Cost. After accounting for retrieval, evidence selection, and model computation, can the approach attain higher accuracy at a fixed resource budget or comparable performance at lower cost? What complementary or substitutive effects arise between text and graph budgets?
RQ4 — Effective Conditions and Failure Modes. How do the benefits vary with question type, context length, knowledge coverage, and local graph structure? When does compression eliminate irrelevant information, and when does it remove essential evidence?
KG-Grounded Language–Graph Models for Task-Agnostic Question Answering (Master Thesis)
QA-GNN [1] and GreaseLM [2] combine textual context with knowledge graph evidence for QA, while DRAGON [3] pretrains joint language–graph representations. ConceptNet [5] provides structured commonsense knowledge, and choice-based decision interfaces such as Jev [4] motivate candidate-defined answer spaces.
Yet joint representation pretraining does not directly establish a reusable candidate-decision function. In particular, it remains unclear how KG relations can provide downstream-task-agnostic decision supervision, whether one model can generalize to previously unseen QA tasks and varying numbers of candidates, and whether its decision probabilities remain reliable. This thesis studies ConceptNet-specific pretraining without target-QA-task supervision, using shared candidate scoring to support transferable decision making.
Research Questions
RQ1 — Knowledge-Driven Decision Learning.How can ConceptNet relations and local structures support task-agnostic decision-pretraining objectives that teach a model to evaluate candidate answers using knowledge evidence? Relative to language-only training and generic self-supervised language–graph pretraining, what produces the observed gains?
RQ2 — Transfer across QA Tasks. Can one pretrained decision model be applied to different commonsense QA tasks without using target-task labels? How is transfer affected by question format, knowledge coverage, and reasoning requirements, and what improvements arise from limited labeled adaptation?
RQ3 — Generalization across Candidate Sets.</strong> Can the model preserve meaningful semantic decisions when the number of options, distractor difficulty, or answer wording changes? How can architectural support for variable-sized option sets be distinguished from empirical generalization to previously unseen candidate sets?
RQ4 — Reliability of Decision Probabilities. Do predicted probabilities reflect actual reliability when answer-option sets and supporting knowledge change? Can task-agnostic training and calibration mitigate overconfidence during cross-task deployment?
Towards a Temporal Heterogeneous Graph Foundation Model for Academic Knowledge Graphs (Master Thesis)
This thesis investigates how to develop a foundation-model-style representation learner for Academic Knowledge Graphs (AKGs), such as arXiv-, DBLP-, MAG-, and Open Academic Graph-derived data. Academic graphs connect papers, authors, venues, institutions, fields of study, keywords, and citations through typed relations that evolve over time, making them a natural setting for temporal heterogeneous graph representation learning [1,9,10].
The thesis focuses on reusable pre-training for scientific knowledge graphs. In particular, it will investigate how structural topology, textual attributes, relation semantics, and temporal information can be integrated within a scalable heterogeneous graph model, and whether the resulting representations transfer across downstream tasks and graph sources [1–8].
Composable Source Descriptions and Declarative Reasoning for Document-Grounded Knowledge Graph Validation
Knowledge graphs built automatically from documents inherit extraction errors: values get truncated, entities are dropped, or records are bound to the wrong document. Structural validators such as SHACL catch formatting violations, but they cannot answer the question that matters for curation: does the source document actually contain the fact the graph claims?
Formal Reasoning over Policy Constraints in the ODRL Multi-Agent LLM Framework
Large Language Models (LLMs) are increasingly used to translate natural-language requirements into machine-readable data usage policies. In the ODRL Multi-Agent LLM framework, such requirements are transformed into ODRL policies through a multi-agent pipeline supported by ontology-guided prompting and structured policy generation [1,2]. However, LLM-based reasoning alone cannot provide formal guarantees about constraint consistency, particularly when policies involve numeric restrictions, temporal conditions, logical combinations of constraints, or semantic relations that depend on external knowledge.
This thesis investigates how an external formal reasoning component can complement LLM-assisted policy creation. ODRL is used as the concrete policy setting because it provides a W3C-standardized information model and vocabulary for permissions, prohibitions, duties, and constraints [3,4]. The thesis focuses initially on self-contained numeric constraints and selected simple temporal constraints that can be translated into formal logical representations and checked using automated solvers. More complex temporal, spatial, purpose-based, language-based, and ontology-grounded constraints are considered possible extensions beyond the initial scope.
A Layered Architecture for Entity Resolution in Scholarly Metadata Combining Algorithmic and Agentic Approaches
Developing an Explainable Anomaly Detection System forSmart Grids by Incorporating Structural and Operational GridKnowledge
Thesis Type
Master
Student: Sebastian Miller
Status
In Progress
Background
Supervisory control and data acquisition (SCADA) systems are increasingly connected through information and communication technologies, exposing smart grids to cyberattacks and operational disruptions. Conventional signature-based intrusion detection systems (IDSs) reliably identify known attacks but cannot detect previously unseen patterns, while statistical and machine-learning-based IDSs may achieve high detection rates but often provide limited transparency and generate false positives. Moreover, both classes frequently struggle with process-related attacks in which individually legitimate commands are executed in an unusual sequence and only become harmful over time. In energy distribution grids, this limitation is particularly critical because operators must not only recognize an anomaly but also understand which component or operational action caused it. The IEC 60870-5-104 communication protocol, together with structural and operational knowledge represented in a Smart Grid Architecture Model (SGAM)-aligned knowledge graph, offers a basis for developing a process-aware and explainable detection approach.
Objectives
The objective of this master’s thesis is to develop and evaluate an explainable, process-aware anomaly detection system for smart grids. The approach combines sequential pattern mining of IEC 60870-5-104 communication data with an RDF-based graph model of grid structure, device roles, communication relationships, and grid-operator actions. The system shall learn frequent sequences from normal operation, identify anomalous or rare sequences in attack scenarios, and use the knowledge graph to indicate likely affected devices and explain deviations from the expected control flow. A secondary objective is to create a reproducible dataset of normal and process-related attack scenarios and to compare the proposed approach with established process-state-aware IDSs through the IPAL framework.
Tasks
The student first conducts a focused literature review on anomaly detection in SCADA and smart-grid environments, explainable intrusion detection, process-aware detection, sequential pattern mining, SGAM-based modeling, and IEC 60870-5-104 traffic analysis. Based on this review, the student derives a precise requirement specification covering the available input data, the required level of event abstraction, explainability criteria, supported attack classes, and evaluation metrics. In parallel, the student becomes familiar with the FIT smart-grid co-simulation environment and the existing SGAM-aligned RDF ontology. Selected grid-operator actions, such as state estimation, feed-in management, transformer tap switching, topology changes, and load shedding, are modeled or linked in the ontology so that expected operational sequences and participating device types can be queried.
The core technical work comprises the creation of simulation scenarios for normal operation and representative process-related attacks, the extraction of IEC 60870-5-104 communication data from PCAP files, and the semantic enrichment of the extracted records. The student maps sender addresses and Information Object Addresses to concrete devices and locations through SPARQL queries, removes redundant status reports, and defines a reproducible event-abstraction scheme for discrete and continuous process values. Based on these events, the student implements a sequential pattern mining method, with particular emphasis on rare sequential patterns, and develops logic for comparing anomalous sequences with frequent reference patterns. The resulting detector shall use graph knowledge to distinguish similar device types, relate events to operator actions, and generate an explanation that highlights deviations and likely affected components.
The approach is implemented as a documented prototype that accepts simulation data and a corresponding knowledge graph, generates event sequences, detects anomalies, and produces human-readable diagnostic output. For evaluation, the student creates a dataset containing training data from normal operation and test data with several attack scenarios. Using IPAL, the prototype is compared with selected process-state-aware IDSs such as PASAD, Seq2Seq-NN, or TABOR. Detection accuracy, false-positive behavior, robustness to different abstraction parameters, and computational effort are assessed quantitatively; the usefulness of the generated explanations and device-level localization is assessed qualitatively. All results are documented, including the ontology extensions, extraction and abstraction rules, algorithm specification, prototype architecture, dataset and scenario definitions, evaluation design, limitations, and recommendations for future online deployment.
Prerequisites
Basic knowledge of IT security, network protocols, and industrial control or SCADA systems; interest in smart grids and cyber-physical energy systems; programming skills, preferably in Python, for PCAP processing and data analysis; and willingness to work with graph-based data models, RDF/SPARQL, and simulation environments. Experience with anomaly detection, pattern mining, IEC 60870-5-104, or semantic-web technologies is helpful but not required. The student should be able to work methodically, document technical decisions clearly, and evaluate a prototype using reproducible experiments.
References (MLA)
- CEN-CENELEC-ETSI Smart Grid Coordination Group. Smart Grid Reference Architecture. 2012.
- Rahman, A., et al. “Finding Anomalies in SCADA Logs Using Rare Sequential Pattern Mining.” International Conference on Network and System Security, Springer, 2016, pp. 499-506.
- Van Der Velde, D., Ö. Sen, and I. Hacker. “Towards a Scalable and Flexible Smart Grid Co-Simulation Environment to Investigate Communication Infrastructures for Resilient Distribution Grid Operation.” 2021 International Conference on Smart Energy Systems and Technologies (SEST), IEEE, 2021, pp. 1-6.
- Wolsing, K., et al. “IPAL: Breaking Up Silos of Protocol-Dependent and Domain-Specific Industrial Intrusion Detection Systems.” Proceedings of the 25th International Symposium on Research in Attacks, Intrusions and Defenses, 2022, pp. 510-525.
- Lin, C.-Y., and S. Nadjm-Tehrani. “Understanding IEC-60870-5-104 Traffic Patterns in SCADA Networks.” Proceedings of the 4th ACM Workshop on Cyber-Physical System Security, 2018, pp. 51-60.
Ontology Reduction for Scalable, Semantic Data Integration
This thesis investigates how compact, application-specific ontology modules can be extracted automatically from large ontologies, preserving the semantic coherence needed for downstream tasks while drastically reducing complexity.
A Comparison of Interaction Modalities for Extended Reality Agents
Currently, the primary way users interact with Large Language Models (LLMs) is through two-dimensional chat interfaces. However, for use cases in Extended Reality (XR) environments, the interaction paradigm shifts from a flat screen to a spatial experience. Here, LLMs can, e.g., be represented as XR agents, a personified version of the LLM. While 3D environments offer high potential for more immersive and intuitive interactions, they also introduce significant challenges regarding user interface design. Simply porting a 2D chat window into a 3D space often feels clunky or breaks the sense of presence, yet purely voice-based interaction may lack the precision or privacy that text provides. This is especially true for tutoring scenarios where the agents need to give precise and memorable instructions. There is currently a lack of systematic research on which interface modalities best support the strengths of LLMs while maintaining the immersion of an XR environment.