DBIS

The thesis project aims to automate the translation of unstructured or semi-structured cybersecurity playbooks into a standardized, machine-readable format (OASIS CACAO) using Large Language Models (LLMs). A key research focus is ensuring the accuracy, reliability, and effectiveness of LLM-generated workflows during playbook translation. It includes a concept where security operators use LLMs to convert unstructured text into structured workflows, with syntax checkers and playbook management components ensuring standard compliance and content accuracy.  The thesis will focus on prompt engineering for state-of-the-art LLM models, e.g, GPT-3 or 4 for the CACAO playbook translation and further development of an already existing CACAO syntax checker component for syntax verification and improving prompting. This will aid in developing a methodological approach to use LLMs for automating the translation of cybersecurity playbooks effectively. 

* OASIS CACAO Specification: This document details the Collaborative Automated Course of Action Operations (CACAO) standard for cybersecurity playbooks: https://docs.oasis-open.org/cacao/security-playbooks/v2.0/security-playbooks-v2.0.html

* CACAO v2.0 syntax validator: https://github.com/opencybersecurityalliance/cacao-roaster/tree/main/src/diagram/modules/features/validator

* Playbook Examples:

1. The link to Phantom Cyber’s GitHub repository will provide simple practical examples of cybersecurity playbooks: https://github.com/phantomcyber/playbooks
2. https://github.com/luduslibrum/awesome-playbooks

* Prompt Engineering with OpenAI: This resource from OpenAI discusses prompt engineering, a crucial aspect for effectively utilizing Large Language Models (LLMs) like GPT-3 or GPT-4. Understanding how to craft prompts that guide LLMs to produce desired outcomes will be key in automating the playbook translation process: https://platform.openai.com/docs/guides/prompt-engineering

Seed Papers:
* https://proceedings.neurips.cc/paper_files/paper/2020/file/1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf
* https://arxiv.org/pdf/2312.16171v1.pdf
* https://openreview.net/pdf?id=gEZrGCozdqR
* https://arxiv.org/pdf/2302.11382.pdf