DBIS

For mixed reality applications, developers usually choose a modular development approach where the application-logic is segmented into separate components which only realize one feature. In well-designed projects, the components are reusable in different contexts within the project but also outside of it. Additionally, code from these projects can be used to find examples how to use APIs or libraries. However, many elementary features of mixed reality applications are re-implemented in new projects, instead of leveraging components from existing open-source projects.

A mixed reality environment immerses users in a three-dimensional space. This allows them to organize digital content in the entire room, e.g. task cards in immersive project management. A natural way to sort related objects is by placing them close to each other. However, this kind of spatial co-location of objects is not enough to stress their relation to each other. If another user who is unaware of the association moves one of the objects, the spatial connection is lost. Hence, lines in 3D space are required which establish a visual and persistent connection between the objects. In a previous project, an initial system was implemented which uses straight lines to connect the objects. However, straight lines have some shortcomings since they intersect with obstacles between the two endpoints. A better connection system can use 3D curves to avoid the obstacles in an efficient way.

Lectures convey 2D content on slides that are projected onto a wall but the understanding of 3D structures is important in anatomy courses or engineering. Moreover, the legibility varies based on the position in the lecture hall due to the angle and distance of the viewer which can influence the learning efficiency. Mixed Reality has the ability to immerse the viewer in the presentation. Since participants can use their own hardware, e.g. smartphones, to view 3D presentations, content can be positioned individually at the optimal location for every person. Additionally, presentations can vary in scale, ranging from miniature 3D models that are displayed on the table of every member in the audience to above-life size close-ups. Lecturers can use the same 3D presentation for remote teaching if face-to-face lectures are not possible.

A challenge of existing educational Mixed Reality applications concerns the difficult accessibility of technology for students. Many applications require expensive head-mounted displays or high-end smartphones which can only be tried by students at the university for a limited amount of time. However, Mixed Reality content can also be shown on a broader range of devices using marker-based technology. Here, a marker is used as an anchor to show a 3D model above it. If the marker is filmed by a smartphone, it can calculate the view angle and render the 3D model from the same perspective. A challenge of marker-based applications is to find a meaningful integration of the markers into the environment. The markers should be easily recognizable but also need to give the user an idea about the 3D model that they can show.

The creation of learning plans is a complex and important task. There are dependencies between the different topics, e.g. an advanced subject can only be understood if the underlying basic theories have been considered. Learning plans contain short-term, mid-term and long-term goals which should ideally be synchronized to each other. Additionally, learning plans bear a strategic component by identifying aspects which are most beneficial for the student’s future. There are also collaborative considerations. Members of learning groups want to adapt their learning plans to each other so that they can either learn topics together simultaneously or so that one group member who is already familiar with the topic is able to support the other group members. A framework for creating learning plans is given by Learning Design which defines clear learning outcomes, activities, times and required resources. Since learning plans are abstract, they are rarely visualized. However, such a visualization can act as a persistent overview that allows students but also lecturers to create consistent learning plans where short-term tasks work towards long-term learning outcomes. To grasp the complex nature of learning plans, a promising solution is the immersive creation of visualizations in 3D using Mixed Reality technology.

Courses in higher education, e.g. at universities, face a challenge regarding scalability. Ideally, every student should be able to contact a mentor to benefit from the mentor’s experience and to reach the learning goals. However, if the number of participants in a course rises, the limited resources of the institution are quickly exhausted. This leads to a high workload for academic staff and decreases the mentoring quality since there is less time for mentors to address the individual needs of the students. A solution to this problem is socio-technical support for mentoring processes which combines social processes like peer mentoring and technological processes, e.g. for student’s feedback. As a result, text-based chat bots were created which can answer student’s questions and give feedback about exercises. We would now like to enhance the interaction with such bots by upgrading them to Mixed Reality agents. Such agents are shown as an avatar in a Mixed Reality environment and can interact with virtual content, users and other bots. They form a natural user interface where students can talk to the agent to get advice and it can also make autonomous decisions to guide the learning process. If the agent cannot answer a question, it can call a human mentor to join the conversation. This system considerably lowers the workload of the mentors while improving the mentoring experience of the students.