EUVIMEDEUVIMED entdecken
QuerschnittstudieOpen Access

Performance of Large Language Models in Patient Complaint Resolution: Web-Based Cross-Sectional Survey

Lorraine Pei Xian Yong iD, Joshua Yi Min Tung iD, Zi Yao Lee iD, Win Sen Kuan iD, Mui Teng Chua iD

Journal of Medical Internet Research · 09.08.2024 · 26 · e56413

DOI: 10.2196/56413 ↗

Kurz gefasst

Querschnittstudie in der Zeitschrift Journal of Medical Internet Research, online erschienen am 09.08.2024. Verfasst von Lorraine Pei Xian Yong und 4 weiteren Autor:innen. Beteiligte Einrichtung: Emergency Medicine Department, National University Hospital, National University Health System, Singapore, Singapore. Hauptthemen laut MEDLINE: Patient Satisfaction. Die Arbeit wurde bisher 16-mal zitiert (Crossref) und nennt 33 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Patient complaints are a perennial challenge faced by health care institutions globally, requiring extensive time and effort from health care workers. Despite these efforts, patient dissatisfaction remains high. Recent studies on the use of large language models (LLMs) such as the GPT models developed by OpenAI in the health care sector have shown great promise, with the ability to provide more detailed and empathetic responses as compared to physicians. LLMs could potentially be used in responding to patient complaints to improve patient satisfaction and complaint response time.

Objective

This study aims to evaluate the performance of LLMs in addressing patient complaints received by a tertiary health care institution, with the goal of enhancing patient satisfaction.

Methods

Anonymized patient complaint emails and associated responses from the patient relations department were obtained. ChatGPT-4.0 (OpenAI, Inc) was provided with the same complaint email and tasked to generate a response. The complaints and the respective responses were uploaded onto a web-based questionnaire. Respondents were asked to rate both responses on a 10-point Likert scale for 4 items: appropriateness, completeness, empathy, and satisfaction. Participants were also asked to choose a preferred response at the end of each scenario.

Results

There was a total of 188 respondents, of which 115 (61.2%) were health care workers. A majority of the respondents, including both health care and non–health care workers, preferred replies from ChatGPT (n=164, 87.2% to n=183, 97.3%). GPT-4.0 responses were rated higher in all 4 assessed items with all median scores of 8 (IQR 7-9) compared to human responses (appropriateness 5, IQR 3-7; empathy 4, IQR 3-6; quality 5, IQR 3-6; satisfaction 5, IQR 3-6; P

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Patient SatisfactionHumansCross-Sectional StudiesFemaleSurveys and QuestionnairesMaleAdultInternetLanguageMiddle AgedElectronic Mail

Stichwörter der Autor:innen

ChatGPTartificial intelligenceefficiencyempathyhealth care complaintlarge language modelspatient complaintpatient satisfactionresource allocation

Autor:innen und Institutionen

  • Emergency Medicine Department, National University Hospital, National University Health System, Singapore, Singapore.
  • Department of Surgery, Yong Loo Lin School of Medicine, National University of Singapore, Singapore, Singapore.
  • Urgent Care Centre, Alexandra Hospital, National University Health System, Singapore, Singapore.
  • Department of Urology, Singapore General Hospital, Singapore, Singapore.

ORCID-Profile: Lorraine Pei Xian Yong, Joshua Yi Min Tung, Zi Yao Lee, Win Sen Kuan, Mui Teng Chua

Zitieren

APA

Yong, L. P. X., Tung, J. Y. M., Lee, Z. Y., Kuan, W. S., & Chua, M. T. (2024). Performance of Large Language Models in Patient Complaint Resolution: Web-Based Cross-Sectional Survey. Journal of Medical Internet Research, 26, e56413. https://doi.org/10.2196/56413

Vancouver

Yong LPX, Tung JYM, Lee ZY, Kuan WS, Chua MT. Performance of Large Language Models in Patient Complaint Resolution: Web-Based Cross-Sectional Survey. Journal of Medical Internet Research. 2024;26:e56413. doi:10.2196/56413

BibTeX
@article{Yong2024,
  author = {Yong, Lorraine Pei Xian and Tung, Joshua Yi Min and Lee, Zi Yao and Kuan, Win Sen and Chua, Mui Teng},
  title = {Performance of Large Language Models in Patient Complaint Resolution: Web-Based Cross-Sectional Survey},
  journal = {Journal of Medical Internet Research},
  year = {2024},
  volume = {26},
  pages = {e56413},
  doi = {10.2196/56413}
}
Kostenlos starten →