Instruction-based enhancement of LLM-assisted cybersecurity playbook translation into a standardized format

June 27th, 2024

Thesis Type
  • Bachelor
Tassilo Jehn
Stefan Decker
Mehdi Akbari G.

The thesis project aims to automate the translation of unstructured or semi-structured cybersecurity playbooks into a standardized, machine-readable format (OASIS CACAO) using Large Language Models (LLMs). A key research focus is ensuring the accuracy, reliability, and effectiveness of LLM-generated workflows during playbook translation. It includes a concept where security operators use LLMs to convert unstructured text into structured workflows, with syntax checkers and playbook management components ensuring standard compliance and content accuracy. The thesis will focus on instruction tuning techniques on state-of-the-art LLM models for the CACAO playbook translation and further development of an already existing CACAO syntax checker component for syntax verification and improving prompting. This will aid in developing a methodological approach to use LLMs for automating the translation of cybersecurity playbooks effectively.

* OASIS CACAO Specification: This document details the Collaborative Automated Course of Action Operations (CACAO) standard for cybersecurity playbooks:

* CACAO v2.0 syntax validator:

* Playbook Examples:

1. The link to Phantom Cyber’s GitHub repository will provide simple practical examples of cybersecurity playbooks:

Seed papers:

Process Modeling With Large Language Models:

A Method for Extracting BPMN Models from Textual Descriptions Using Natural Language Processing:

Principled Instructions Are All You Need for Questioning LLaMA-1/2, GPT-3.5/4:


Basic knowledge in the domains of cyber security, Natural Language Processing (Specifically, Generative AI).