Enhancing Chatbot Performance through Process Mining

March 6th, 2023

This thesis aims to develop a framework for analyzing chatbot conversations using process mining techniques. The framework will allow users to discover process models from chatbot event logs, compare them to the bot models, and enhance them based on the analysis results.

Thesis Type
  • Master
Ben Lakhoune
Submitted in
Proposal on
04/07/2023 5:00 pm
Proposal room
Presentation on
05/03/2024 10:30 am
Presentation room
Seminar room I5 6202
Stefan Decker
Alexander Neumann

This thesis aims to develop a framework for analyzing and improving the performance of chatbots using process mining techniques. Chatbots have become increasingly popular in various domains, such as customer service and education, but their effectiveness can be hindered by factors like misunderstood user intents and inconsistent training data. The framework should address these challenges by integrating process mining concepts with chatbot analytics.

The framework should start by modeling the conversation paths of the chatbot, referred to as the bot model. This model represents the intended flow of the conversation and captures the different activities and events that can occur during an interaction.

An event log should be generated from real-world user conversations to analyze the chatbot’s performance. The event log should include user messages, bot responses, and background processes such as intent recognition and API calls. These logs should then be used as input for process mining techniques to discover a process model that represents the actual flow of the conversation.

Further comparison with the bot model should identify deviations and inconsistencies. This comparison will evaluate how well the chatbot meets user expectations and intentions.

To further improve the chatbot, the framework should propose techniques to enhance the bot model. Adding frequency and temporal information to the process model could provide insights into the most frequent conversation paths and bottlenecks. Additionally, the framework should consider using large pre-trained language models (LLMs) to enhance the chatbot’s performance.