DBIS

Technical support is an essential aspect of various industries, e.g., to provide help with maintaining machinery and IT systems. However, diagnosing error messages and faults in complex technologies can be a time-consuming and challenging task. The maintainer has to search through the long documentation booklets for the technology in order to find a solution or wait for an appointment so that the manufacturer sends an expert. A combination of mixed reality (MR) agents and large language models (LLMs) offer potential to provide quick and interactive diagnosis suggestions while providing visual help. For instance, the MR agent could be placed next to the faulty machine and embody the LLM. The technical person could talk to the agent which, in turn, forwards the inputs to an LLM. Through techniques such as retrieval augmented generation (RAG), the LLM could match the described problems with suitable excerpts of the documentation. Thus, it can quickly identify relevant sections in a long document and provide help based on the knowledge that is provided in the documentation. The answer could then be output by the MR agent and it could be combined with pre-defined actions by the MR agent, e.g., to point to specific points on the machine or to demonstrate a process on a virtual replica.