Reasoning-based LLM enhancement for the cybersecurity playbook translation into a standardised format

June 27th, 2024

Thesis Type
  • Bachelor
Justin Backes
Proposal on
12/07/2024 10:45 am
Proposal room
Seminar room I5 6202
Stefan Decker
Mehdi Akbari G.

The thesis project aims to automate the translation of unstructured or semi-structured cybersecurity playbooks into a standardized, machine-readable format (OASIS CACAO) using Large Language Models (LLMs). A key research focus is ensuring the accuracy, reliability, and effectiveness of LLM-generated workflows during playbook translation. It includes a concept where security operators use LLMs to convert unstructured text into structured workflows, with syntax checkers and playbook management components ensuring standard compliance and content accuracy. The thesis will focus on reasoning-based techniques such as chain of Thought and React, on state-of-the-art LLM models for the CACAO playbook translation and further development of an already existing CACAO syntax checker component for syntax verification and improving prompting. This will aid in developing a methodological approach to use LLMs for automating the translation of cybersecurity playbooks effectively.

* OASIS CACAO Specification: This document details the Collaborative Automated Course of Action Operations (CACAO) standard for cybersecurity playbooks:

* CACAO v2.0 syntax validator:

* Playbook Examples:

1. The link to Phantom Cyber’s GitHub repository will provide simple practical examples of cybersecurity playbooks:

Seed papers:

Process Modeling With Large Language Models:

A Method for Extracting BPMN Models from Textual Descriptions Using Natural Language Processing:

Chain-Of-Knowledge: Grounding Large Language Models Via Dynamic Knowledge Adapting Over Heterogeneous Sources”:


Basic knowledge in the domains of cyber security, Natural Language Processing (Specifically, Generative AI).