EUVIMEDEUVIMED entdecken
Open Access

Using ChatGPT for human–computer interaction research: a primer

Wilbert Tabone iD, Joost de Winter iD

Royal Society Open Science · 2023-09 · 10 (9)

DOI: 10.1098/rsos.231053 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Royal Society Open Science, online erschienen am 13.09.2023. Verfasst von Wilbert Tabone und 1 weiteren Autor:innen. Beteiligte Einrichtung: Department of Cognitive Robotics, Faculty of Mechanical, Maritime and Materials Engineering, Delft University of Technology, Delft 2628CD, The Nether…. Die Arbeit wurde bisher 107-mal zitiert (Crossref) und nennt 82 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

ChatGPT could serve as a tool for text analysis within the field of Human–Computer Interaction, though its validity requires investigation. This study applied ChatGPT to: (1) textbox questionnaire responses on nine augmented-reality interfaces, (2) interview data from participants who experienced these interfaces in a virtual simulator, and (3) transcribed think-aloud data of participants who viewed a real painting and its replica. Using a hierarchical approach, ChatGPT produced scores or summaries of text batches, which were then aggregated. Results showed that (1) ChatGPT generated sentiment scores of the interfaces that correlated extremely strongly ( r > 0.99) with human rating scale outcomes and with a rule-based sentiment analysis method (criterion validity). Additionally, (2) by inputting automatically transcribed interviews to ChatGPT, it provided meaningful meta-summaries of the qualities of the interfaces (face validity). One meta-summary analysed in depth was found to have substantial but imperfect overlap with a content analysis conducted by an independent researcher (criterion validity). Finally, (3) ChatGPT's summary of the think-aloud data highlighted subtle differences between the real painting and the replica (face validity), a distinction corresponding with a keyword analysis (criterion validity). In conclusion, our research indicates that, with appropriate precautions, ChatGPT can be used as a valid tool for analysing text data.

Schlagwörter

Stichwörter der Autor:innen

application programming interface (API)human-subject researchprompt engineeringreproducibility

Autor:innen und Institutionen

  • Department of Cognitive Robotics, Faculty of Mechanical, Maritime and Materials Engineering, Delft University of Technology, Delft 2628CD, The Netherlands.

ORCID-Profile: Wilbert Tabone, Joost de Winter

Zitieren

APA

Tabone, W., & de Winter, J. (2023). Using ChatGPT for human–computer interaction research: a primer. Royal Society Open Science, 10(9). https://doi.org/10.1098/rsos.231053

Vancouver

Tabone W, de Winter J. Using ChatGPT for human–computer interaction research: a primer. Royal Society Open Science. 2023;10(9). doi:10.1098/rsos.231053

BibTeX
@article{Tabone2023,
  author = {Tabone, Wilbert and de Winter, Joost},
  title = {Using ChatGPT for human–computer interaction research: a primer},
  journal = {Royal Society Open Science},
  year = {2023},
  volume = {10},
  number = {9},
  doi = {10.1098/rsos.231053}
}
Kostenlos starten →