DBIS

This thesis aims to develop a framework for analyzing and improving the performance of chatbots using process mining techniques. Chatbots have become increasingly popular in various domains, such as customer service and education, but their effectiveness can be hindered by factors like misunderstood user intents and inconsistent training data. The framework should address these challenges by integrating process mining concepts with chatbot analytics.

The framework should start by modeling the conversation paths of the chatbot, referred to as the bot model. This model represents the intended flow of the conversation and captures the different activities and events that can occur during an interaction.

An event log should be generated from real-world user conversations to analyze the chatbot’s performance. The event log should include user messages, bot responses, and background processes such as intent recognition and API calls. These logs should then be used as input for process mining techniques to discover a process model that represents the actual flow of the conversation.

Further comparison with the bot model should identify deviations and inconsistencies. This comparison will evaluate how well the chatbot meets user expectations and intentions.

To further improve the chatbot, the framework should propose techniques to enhance the bot model. Adding frequency and temporal information to the process model could provide insights into the most frequent conversation paths and bottlenecks. Additionally, the framework should consider using large pre-trained language models (LLMs) to enhance the chatbot’s performance.