Exploring Information Interaction Preferences in an LLM-Assisted Learning Environment with a Topic Modeling Framework
Applied Sciences · 04.07.2025 · 15 (13) · 7515
DOI: 10.3390/app15137515 ↗Kurz gefasst
Fachartikel in der Zeitschrift Applied Sciences, online erschienen am 04.07.2025. Verfasst von Yiming Taclis Luo und 4 weiteren Autor:innen. Die Arbeit wurde bisher 12-mal zitiert (Crossref) und nennt 80 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Macao Polytechnic University research grant.
Abstract (Originalsprache)
Large Language Models (LLMs) are driving a revolution in the way we access information, yet there remains a lack of exploration to capture people’s information interaction preferences in LLM environments. In this study, we designed a comprehensive analysis framework to evaluate students’ prompt texts during a professional academic writing task. The framework includes a dimensionality reduction and classification method, three topic modeling approaches, namely BERTopic, BoW-LDA, and TF-IDF-NMF, and a set of evaluation criteria. These criteria assess both the semantic quality of topic content and the structural quality of clustering. Using this framework, we analyzed 288 prompt texts to identify key topics that reflect students’ information interaction behaviors. The results showed that students with low academic performance tend to focus on structural clarity and task execution, including task inquiry, format specifications, and methodological search, indicating that their interaction mode is instruction-oriented. In contrast, students with high academic performance interact with LLM not only in basic task completion but also in knowledge integration and the pursuit of novel ideas. This is reflected in more complex topic levels and diverse, innovative keywords. It shows that they have stronger self-planning and self-regulation abilities. This study provides a new approach to studying the interaction between students and LLM in engineering education by using natural language processing to process prompts, contributing to the exploration of the performance of students with different performance levels in professional academic writing using LLM.
Förderung
- Macao Polytechnic University research grant
Zitieren
APA
Luo, Y. T., Liu, T., Pang, P. C., Wang, Z., & Chan, K. I. (2025). Exploring Information Interaction Preferences in an LLM-Assisted Learning Environment with a Topic Modeling Framework. Applied Sciences, 15(13), 7515. https://doi.org/10.3390/app15137515
Vancouver
Luo YT, Liu T, Pang PC, Wang Z, Chan KI. Exploring Information Interaction Preferences in an LLM-Assisted Learning Environment with a Topic Modeling Framework. Applied Sciences. 2025;15(13):7515. doi:10.3390/app15137515
BibTeX
@article{Luo2025,
author = {Luo, Yiming Taclis and Liu, Ting and Pang, Patrick Cheong-Iao and Wang, Zhuo and Chan, Ka Ian},
title = {Exploring Information Interaction Preferences in an LLM-Assisted Learning Environment with a Topic Modeling Framework},
journal = {Applied Sciences},
year = {2025},
volume = {15},
number = {13},
pages = {7515},
doi = {10.3390/app15137515}
}
EUVIMED