The increasing connectivity of Industrial Control Systems (ICS) has elevated the need for robust cybersecurity measures. However, evaluating the effectiveness of Intrusion Detection Systems (IDS) in ICS environments remains fragmented and inconsistent. This thesis addresses this challenge by developing a systematic, modular benchmarking environment that enables reproducible and standardized evaluation of machine learning-based IDS across diverse datasets. By introducing a unified data format and structured evaluation protocol, the work aims to enhance the comparability, transparency, and practical relevance of ICS security research.
Thesis Type |
|
Student |
Hanzala Gulraiz |
Status |
Running |
Presentation room |
Seminar room I5 6202 |
Supervisor(s) |
Stefan Decker |
Advisor(s) |
osen |
Contact |
oemer.sen@fit.fraunhofer.de |
The thesis focuses on closing critical gaps in the evaluation of IDS for industrial control systems by designing a dedicated benchmarking pipeline. Key elements of the approach include:
-
Unified Data Representation: Heterogeneous datasets are converted into a standardized format, ensuring consistent processing and analysis across different ICS and network domains.
-
Modular Benchmarking Pipeline: The environment is structured into clear stages, data ingestion, preprocessing, model training, evaluation, and visualization, each implemented to maximize reproducibility and extendibility.
-
Diverse Dataset Integration: The benchmark incorporates multiple datasets, representing different attack surfaces and operational scenarios.
-
Model Evaluation Framework: A variety of supervised machine learning approaches, and ensemble methods, are systematically assessed using consistent training and evaluation protocols.
-
User-Centric Validation: The benchmarking environment is validated against three operationally relevant user stories, ICS operators prioritizing low false positives, researchers comparing models, and developers optimizing detection performance.
The result is a comprehensive, reusable framework that supports transparent comparison of IDS models, enhances practical applicability, and fosters scientific rigor in cybersecurity evaluations for smart grids and ICS environments.
To successfully conduct work based on this benchmarking environment, the following background is recommended:
-
Cybersecurity Fundamentals
-
Understanding of ICS-specific threat landscapes and cybersecurity priorities (e.g., availability, integrity).
-
Familiarity with common attack scenarios in smart grids and industrial settings.
-
-
Machine Learning Expertise
-
Practical experience with supervised learning methods (e.g., Random Forest, Neural Networks).
-
Knowledge of ensemble learning and model evaluation metrics (e.g., F1-score, ROC-AUC).
-
-
Data Handling and Programming
-
Proficiency in Python, including libraries for data processing (pandas, NumPy), machine learning (scikit-learn, TensorFlow/Keras), and visualization (Matplotlib, Seaborn).
-
Skills in dataset preprocessing, feature engineering, and data normalization.
-
-
Benchmarking and Evaluation
-
Understanding of systematic benchmarking principles: reproducibility, consistency, and comparative analysis.
-
Familiarity with metrics-driven evaluation and visualization techniques for model assessment.
-
-
Project and Research Skills
-
Capability to design, document, and interpret structured experiments.
-
Critical thinking skills to analyze model limitations and benchmarking resul
-