DBIS

Recent LLM-based quality-assessment approaches address this by reading the source documents against the ontology and letting the model judge whether a record is supported. This works well where the mapping from document to graph is direct, but the conventions that govern it are usually left implicit: which document types are expected to witness a given property, how evidence from several documents should be combined, and which notational differences between source and graph are conventional rather than erroneous. When these conventions live only in the prompt or in the model’s judgement, results are harder to reproduce and to attribute to a specific cause. In data integration such knowledge is formalised as source descriptions, but the concept has mostly been developed for structured sources rather than unstructured documents. Existing approaches also tend to combine reading the document and deciding on the defect in a single model step, which makes it difficult to separate extraction quality from reasoning quality in an evaluation.

This thesis develops a declarative source description for documents, written per document type and composable so that several documents contribute overlapping witness sets for one record, together with a validator that separates the two steps. A language model is used only to extract what a document states about a property; every validation judgement is then derived by a Datalog-style logic engine whose rules are instantiated from the source descriptions. Descriptions thereby receive a formal meaning as the logic program they compile to, and composition across document types becomes well-defined.

Research Questions

• How must a declarative source description be designed, which conventions must it express (witnessing, authority, exhaustiveness, tolerances), and in what form, so that document-grounded validation is well-posed and descriptions compose across document types?
• Does an explicit source description improve validation over an agent given the same ontology, records and documents without one, and how much of any improvement stems from the conventions themselves, from their machine-readable form, and from moving the combination and judgement logic out of the model into a deterministic reasoner?
• Which defect classes are reachable by deterministic reasoning alone and which genuinely require semantic interpretation of the source?
• How small can the extraction model be while still providing useful validation results, and are its confidence estimates calibrated well enough to prioritise records for manual review?

Tasks

• Literature review on knowledge graph quality assessment, source descriptions and declarative mappings, LLM-based document verification, and Datalog reasoning with provenance
• Design of the source description vocabulary and its compilation to a logic program, instantiated for CEUR-WS
• Implementation of the proof-of-concept validator with structured extraction and reasoner-derived findings
• Ground-truth construction and evaluation: detection quality, ablation, calibration, model size