EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Applying Large Language Model to User Experience Testing

Nien-Lin Hsueh, Hsuen-Jen Lin, Lien-Chi Lai

Electronics · 24.11.2024 · 13 (23) · 4633

DOI: 10.3390/electronics13234633 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Electronics, online erschienen am 24.11.2024. Verfasst von Nien-Lin Hsueh und 2 weiteren Autor:innen. Die Arbeit wurde bisher 17-mal zitiert (Crossref) und nennt 16 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Science and Technology Council, Taiwan R.O.C..

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The maturation of internet usage environments has elevated User Experience (UX) to a critical factor in system success. However, traditional manual UX testing methods are hampered by subjectivity and lack of standardization, resulting in time-consuming and costly processes. This study explores the potential of Large Language Models (LLMs) to address these challenges by developing an automated UX testing tool. Our innovative approach integrates the Rapi web recording tool to capture user interaction data with the analytical capabilities of LLMs, utilizing Nielsen’s usability heuristics as evaluation criteria. This methodology aims to significantly reduce the initial costs associated with UX testing while maintaining assessment quality. To validate the tool’s efficacy, we conducted a case study featuring a tennis-themed course reservation system. The system incorporated multiple scenarios per page, allowing users to perform tasks based on predefined goals. We employed our automated UX testing tool to evaluate screenshots and interaction logs from user sessions. Concurrently, we invited participants to test the system and complete UX questionnaires based on their experiences. Comparative analysis revealed that varying prompts in the automated UX testing tool yielded different outcomes, particularly in detecting interface elements. Notably, our tool demonstrated superior capability in identifying issues aligned with Nielsen’s usability principles compared to participant evaluations. This research contributes to the field of UX evaluation by leveraging advanced language models and established usability heuristics. Our findings suggest that LLM-based automated UX testing tools can offer more consistent and comprehensive assessments.

Förderung

  • National Science and Technology Council, Taiwan R.O.C.

Zitieren

APA

Hsueh, N., Lin, H., & Lai, L. (2024). Applying Large Language Model to User Experience Testing. Electronics, 13(23), 4633. https://doi.org/10.3390/electronics13234633

Vancouver

Hsueh N, Lin H, Lai L. Applying Large Language Model to User Experience Testing. Electronics. 2024;13(23):4633. doi:10.3390/electronics13234633

BibTeX
@article{Hsueh2024,
  author = {Hsueh, Nien-Lin and Lin, Hsuen-Jen and Lai, Lien-Chi},
  title = {Applying Large Language Model to User Experience Testing},
  journal = {Electronics},
  year = {2024},
  volume = {13},
  number = {23},
  pages = {4633},
  doi = {10.3390/electronics13234633}
}
Kostenlos starten →