In-depth investigation into Catastrophic Forgetting in Distributed Analytics using Model Averaging

August 26th, 2022

Thesis Type
  • Bachelor
Stefan Decker
Sascha Welten

Applying deep learning in the medical domain can become challenging due to the distributed institutional availability of medical
data and corresponding privacy concerns. Through two different approaches, Federated Learning (FL) and Cyclic Institutional Incremental
Learning (CIIL), the field of Distributed Analytics (DA) aims to alleviate such concerns by bringing the training algorithm to the data instead of
vice-versa. However, especially incremental learning is susceptible to the Catastrophic Forgetting (CF) phenomenon, which is noticeable by periodical performance degradation
and training instability during CIIL. Previous work has proposed multiple measures aimed at reducing the negative effects of this phenomenon.

In this work, we review and apply multiple countermeasures and develop a custom mitigation strategy inspired by replay methods and model averaging.
We investigate CF during CIIL, and determine the influence of data set size and data distribution to this problem.

The goal of this thesis is the development of a sophisticated measure against CF based on preliminary works and its influence on different data sets.

If you are interested in this thesis, do not hesitate to contact us via
Please attach your CV and your current gradings.


Machine Learning (Python, ML Libs)
Distributed Analytics