Near Real-time Anomaly Detection from Videostreams using WASM and Machine Learning

May 12th, 2023

Detect Flames in industrial Video data using edge devices.

Thesis Type
  • Master
Sandra Geisler
Liam Tirpitz

In Industrial Manufacturing, the quality of produced parts can sometimes be predicted from visually inspecting the manufacturing process. Experts with domain knowledge can determine anomalies in the production by visual observation. Alternatively, machine learning is used to automatically detect certain conditions from video data. Since video cameras generate large amounts of video data, it is desirable to process the data stream as close to the data source as possible instead of sending it to the cloud. Therefore, cheap and simple edge devices (NVIDIA Jetson) can be used to locally analyze video streams instead of sending the data to more powerful cloud environments. However, modern manufacturing environments are complex and may consist of a large variety of machines, data sources, and data processing devices.
To orchestrate the data processing in such a heterogeneous edge environment, Data Stream Processing on the Edge (DSPoE) systems have emerged.
In our research, we build a language-, platform-, and hardware independent DSPoE architecture to enable flexible stream processing in such environments, using WebAssembly (Wasm) and contribute to industrial use cases in context of the cluster of excellence “Internet of Production” (IoP).

Pouring process without lubricant combustion in the shot sleeve

Pouring process with lubricant combustion in the shot sleeve

During this thesis, you should integrate machine learning operators into the existing DSPoE architecture with WASM and NVIDIA Jetson edge devices. This operator should be subsequently evaluated with industrial video data from a High Pressure Die Casting Process from the Foundry Institute of RWTH Aachen University to detect anomalies such as undesirable (usually by excess lubricant induced) combustion processes at the pour hole.

Full Thesis Description as PDF

Further Information:

  • Good programming skills (Rust is an advantage, but not required)
  • Some experience (from lectures or otherwise) with at least one of the following topics
    • Data Stream Processing
    • Machine Learning
    • Wasm