As universities strive to enhance the effectiveness of their lecture exercises, there arises a need for diverse and realistic test user scenarios to evaluate the understandability and usefulness of educational materials. However, in creating such scenarios a number of challenges arise: Real world students can rarely be used for testing, they are likely inexperienced or experienced but have little incentive to refresh their knowledge to test tasks for lectures. In addition, leaking tasks to students can lead to skewed learning results during the actual course. On the other hand, simple unit tests can neglect the importance of a task being understandable to a human and do not offer much insight into task difficulty.
Thesis Type |
|
Status |
Running |
Presentation room |
Seminar room I5 6202 |
Supervisor(s) |
Stefan Decker |
Advisor(s) |
Laurenz Neumann Maximilian Kißgen |
Contact |
laurenz.neumann@dbis.rwth-aachen.de kissgen@dbis.rwth-aachen.de |
Thesis Objective: This Bachelor’s thesis aims to investigate the feasibility and effectiveness of LLMs posing as artificial test users for university lecture exercises. The goal is to create personas capable of engaging with educational content in a manner reflective of real-world student behaviour.
Thesis Goals:
- Developing pipelines to generate LLM test users based on technical documentation and lecture content.
- Assessing the adaptability of LLM-generated test users across different subject domains.
- Evaluating the usefulness of LLM test users for providing feedback on educational content and preventing possible misunderstanding of tasks.
- Generating reproducible LLM test user personas for reliable testing.
- Background in machine learning, natural language processing, and/or educational technology.
- Proficiency in programming languages such as Python and experience with relevant libraries (e.g., TensorFlow, PyTorch).
- Experience with Jupiter notebooks and CI/CD.