DBIS

A mixed reality environment immerses users in a three-dimensional space. This allows them to organize digital content in the entire room, e.g. task cards in immersive project management. A natural way to sort related objects is by placing them close to each other. However, this kind of spatial co-location of objects is not enough to stress their relation to each other. If another user who is unaware of the association moves one of the objects, the spatial connection is lost. Hence, lines in 3D space are required which establish a visual and persistent connection between the objects. In a previous project, an initial system was implemented which uses straight lines to connect the objects. However, straight lines have some shortcomings since they intersect with obstacles between the two endpoints. A better connection system can use 3D curves to avoid the obstacles in an efficient way.

Lectures convey 2D content on slides that are projected onto a wall but the understanding of 3D structures is important in anatomy courses or engineering. Moreover, the legibility varies based on the position in the lecture hall due to the angle and distance of the viewer which can influence the learning efficiency. Mixed Reality has the ability to immerse the viewer in the presentation. Since participants can use their own hardware, e.g. smartphones, to view 3D presentations, content can be positioned individually at the optimal location for every person. Additionally, presentations can vary in scale, ranging from miniature 3D models that are displayed on the table of every member in the audience to above-life size close-ups. Lecturers can use the same 3D presentation for remote teaching if face-to-face lectures are not possible.

A challenge of existing educational Mixed Reality applications concerns the difficult accessibility of technology for students. Many applications require expensive head-mounted displays or high-end smartphones which can only be tried by students at the university for a limited amount of time. However, Mixed Reality content can also be shown on a broader range of devices using marker-based technology. Here, a marker is used as an anchor to show a 3D model above it. If the marker is filmed by a smartphone, it can calculate the view angle and render the 3D model from the same perspective. A challenge of marker-based applications is to find a meaningful integration of the markers into the environment. The markers should be easily recognizable but also need to give the user an idea about the 3D model that they can show.

The creation of learning plans is a complex and important task. There are dependencies between the different topics, e.g. an advanced subject can only be understood if the underlying basic theories have been considered. Learning plans contain short-term, mid-term and long-term goals which should ideally be synchronized to each other. Additionally, learning plans bear a strategic component by identifying aspects which are most beneficial for the student’s future. There are also collaborative considerations. Members of learning groups want to adapt their learning plans to each other so that they can either learn topics together simultaneously or so that one group member who is already familiar with the topic is able to support the other group members. A framework for creating learning plans is given by Learning Design which defines clear learning outcomes, activities, times and required resources. Since learning plans are abstract, they are rarely visualized. However, such a visualization can act as a persistent overview that allows students but also lecturers to create consistent learning plans where short-term tasks work towards long-term learning outcomes. To grasp the complex nature of learning plans, a promising solution is the immersive creation of visualizations in 3D using Mixed Reality technology.

Courses in higher education, e.g. at universities, face a challenge regarding scalability. Ideally, every student should be able to contact a mentor to benefit from the mentor’s experience and to reach the learning goals. However, if the number of participants in a course rises, the limited resources of the institution are quickly exhausted. This leads to a high workload for academic staff and decreases the mentoring quality since there is less time for mentors to address the individual needs of the students. A solution to this problem is socio-technical support for mentoring processes which combines social processes like peer mentoring and technological processes, e.g. for student’s feedback. As a result, text-based chat bots were created which can answer student’s questions and give feedback about exercises. We would now like to enhance the interaction with such bots by upgrading them to Mixed Reality agents. Such agents are shown as an avatar in a Mixed Reality environment and can interact with virtual content, users and other bots. They form a natural user interface where students can talk to the agent to get advice and it can also make autonomous decisions to guide the learning process. If the agent cannot answer a question, it can call a human mentor to join the conversation. This system considerably lowers the workload of the mentors while improving the mentoring experience of the students.

Mixed reality provides many opportunities for learning applications in formal and informal education. In previous projects, we created different learning applications for various use cases. The applications target technologies like the Microsoft HoloLens, HTC Vive, Oculus Quest 2 and smartphones. Initial pilot evaluations of these applications have already been conducted. However, we would like to have a more detailed and systematic analysis of the application’s characteristics in their intended use cases.

A common problem with spatial immersive applications is the placement of UI elements in the space. On the one hand, a menu needs to be accessible and on the other hand, it must not be in the way of the user when it is not needed. Moreover, the menus need to adapt to the room’s characteristics, e.g. to avoid problems where menus clip through walls and become unreachable. For mixed reality applications, the environment can vary from small rooms to large lecture halls. UI systems must account for this. For instance, UI elements cannot be placed in a fixed position in space without respecting the spatial mapping of the real environment. Instead, a more intelligent solution is required for positioning UI elements in space.

Existing Virtual Learning Environments like Moodle include different multi-media modules to allow lecturers to convey the learning material in a suitable way. However, Virtual Learning Environments have not yet adapted Mixed Reality technology despite its large potential to support learning processes.

Collaboration and group work are essential parts of learning since they encourage students to exchange ideas, to learn from each other and to support each other. The ability to work as a team increases the quality and amount of results and is therefore a highly demanded soft skill. However, students who face group work in courses still encounter challenges regarding efficient coordination, motivation and communication when working on group assignments or in learning groups. This becomes evident in processes where one person does all the work. Other common examples include a lack of communication where the group accumulates their individual results only at the end of the project and the participants cannot compensate if some members failed to deliver their results on time. Mixed Reality bears the potential to provide a structured 3D environment for organization and collaboration where group tasks can be split up visually between participants in virtual meetings. Additionally, Gamification can provide means of motivation to continue with a learning task and to strengthen the unity of the team.