Development of an Assessment Scale for Measurement of Usability and User Experience Characteristics of Bing Chat Conversational AI
Future Internet · 23.12.2023 · 16 (1) · 4
DOI: 10.3390/fi16010004 ↗Kurz gefasst
Fachartikel in der Zeitschrift Future Internet, online erschienen am 23.12.2023. Verfasst von Goran Bubaš und 2 weiteren Autor:innen. Die Arbeit wurde bisher 28-mal zitiert (Crossref) und nennt 70 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
After the introduction of the ChatGPT conversational artificial intelligence (CAI) tool in November 2022, there has been a rapidly growing interest in the use of such tools in higher education. While the educational uses of some other information technology (IT) tools (including collaboration and communication tools, learning management systems, chatbots, and videoconferencing tools) have been frequently evaluated regarding technology acceptance and usability attributes of those technologies, similar evaluations of CAI tools and services like ChatGPT, Bing Chat, and Bard have only recently started to appear in the scholarly literature. In our study, we present a newly developed set of assessment scales that are related to the usability and user experiences of CAI tools when used by university students, as well as the results of evaluation of these assessment scales specifically regarding the CAI Bing Chat tool (i.e., Microsoft Copilot). The following scales were developed and evaluated using a convenience sample (N = 126) of higher education students: Perceived Usefulness, General Usability, Learnability, System Reliability, Visual Design and Navigation, Information Quality, Information Display, Cognitive Involvement, Design Appeal, Trust, Personification, Risk Perception, and Intention to Use. For most of the aforementioned scales, internal consistency (Cronbach alpha) was in the range from satisfactory to good, which implies their potential usefulness for further studies of related attributes of CAI tools. A stepwise linear regression revealed that the most influential predictors of Intention to Use Bing Chat (or ChatGPT) in the future were the usability variable Perceived Usefulness and two user experience variables—Trust and Design Appeal. Also, our study revealed that students’ perceptions of various specific usability and user experience characteristics of Bing Chat were predominantly positive. The evaluated assessment scales could be beneficial in further research that would include other CAI tools like ChatGPT/GPT-4 and Bard.
Zitieren
APA
Bubaš, G., Čižmešija, A., & Kovačić, A. (2023). Development of an Assessment Scale for Measurement of Usability and User Experience Characteristics of Bing Chat Conversational AI. Future Internet, 16(1), 4. https://doi.org/10.3390/fi16010004
Vancouver
Bubaš G, Čižmešija A, Kovačić A. Development of an Assessment Scale for Measurement of Usability and User Experience Characteristics of Bing Chat Conversational AI. Future Internet. 2023;16(1):4. doi:10.3390/fi16010004
BibTeX
@article{Buba2023,
author = {Bubaš, Goran and Čižmešija, Antonela and Kovačić, Andreja},
title = {Development of an Assessment Scale for Measurement of Usability and User Experience Characteristics of Bing Chat Conversational AI},
journal = {Future Internet},
year = {2023},
volume = {16},
number = {1},
pages = {4},
doi = {10.3390/fi16010004}
}
EUVIMED