Categories
Pages
-

DBIS

Immersive Vocabulary Learning with Large Language Models

November 19th, 2024

The emergence of large language models (LLMs), along with recent advances in mixed reality (MR) and virtual reality (VR), enable new opportunities for applying virtual agents in education. These simulated humans can imitate real-life situations and interactions with native speakers, which leads to an immersive and engaging learning experience. Especially in VR, interactions can be simulated with the agents. This way, languages can be learned and practiced in realistic scenarios. The LLM has the potential to overcome the limitations of learning applications with pre-scripted scenarios as the LLM can react dynamically to the learner’s actions and can lead to personalized interactions.

Thesis Type
  • Bachelor
Student
Lynn Nati
Status
Running
Presentation room
Seminar room I5 6202
Supervisor(s)
Stefan Decker
Advisor(s)
Benedikt Hensen
Contact
hensen@dbis.rwth-aachen.de

The goal of this thesis is to investigate a suitable design and integration of LLM-driven virtual agents into language learning and to measure their effectiveness for learning and student motivation. This involves developing an VR learning application which contains virtual agents based on our Virtual Agents Framework (https://github.com/rwth-acis/Virtual-Agents-Framework). An LLM should be connected to the immersive environment so that it can participate in the interactions. As such a technical challenge of the thesis is the integration of the LLM with the virtual agents and finding suitable prompts that steer the answers of the LLM. On the scientific level, a user study needs to be conducted to assess the effectiveness and performance of the LLM-driven virtual agents, e.g., by comparing them to traditional language learning methods like flashcards.

 

The outcomes of the thesis will be an LLM-driven VR language learning system that demonstrates the potential to improve language learning and student engagement, and an accompanying user study with the implemented system. The results of the user study can provide valuable insights into the innovative approach and pave the way for future research in this area.


Prerequisites:

Must: Experience with the Unity 3D engine and C#

Beneficial: Knowledge about LLMs