DBIS

Conveying information in a tour traditionally either requires a real human or an audio recording. However, the human guide might not always be available and the quality in the tours varies. With audio recordings, extra care has to be taken to convey to the user which element is currently being talked about since there are no visual clues. With recent advancements in mobile augmented reality (AR), another option becomes viable where mixed reality (MR) agents can point out landmarks, give directions, and provide guidance based on location and surroundings, making the user’s experience immersive and interactive. MR agents are virtual, human-like entities interacting with users within augmented or virtual environments, blending digital content with the real world. Large Language Models (LLMs) can add conversational intelligence to the experience. They understand natural language inputs, answer questions, and provide detailed, personalized information based on user preferences. Together, MR agents and LLMs could create a system where the agent visually guides users through real-world environments. At the same time, the LLM delivers rich, context-specific information, responds to queries, and adapts content dynamically to the user. In the context of a tour guide, this integration enables users to explore cities or other locations with the MR agent directing them and visually pointing out features, while the LLM provides detailed explanations, answers, and personalized insights.