EUVIMEDEUVIMED entdecken
Vergleichsstudie

Comparing the quality of ChatGPT- and physician-generated responses to patients’ dermatology questions in the electronic medical record

Kelly Reynolds, Daniel Nadelman, Joseph Durgin, Stephen Ansah-Addo, Daniel Cole, Rachel Fayne, Jane Harrell, Madison Ratycz, Mason Runge, Amanda Shepard-Hayes, Daniel Wenzel, Trilokraj Tejasvi

Clinical and Experimental Dermatology · 2024

DOI: 10.1093/ced/llad456 ↗

Kurz gefasst

Vergleichsstudie in der Zeitschrift Clinical and Experimental Dermatology, 2024. Verfasst von Kelly Reynolds und 11 weiteren Autor:innen. Beteiligte Einrichtung: Department of Dermatology, University of Michigan, Ann Arbor, MI, USA. Hauptthemen laut MEDLINE: Electronic Health Records, Dermatology, Natural Language Processing. Die Arbeit wurde bisher 28-mal zitiert (Crossref) und nennt 21 Literaturangaben.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

ChatGPT is a free artificial intelligence (AI)-based natural language processing tool that generates complex responses to inputs from users.

Objectives

To determine whether ChatGPT is able to generate high-quality responses to patient-submitted questions in the patient portal.

Methods

Patient-submitted questions and the corresponding responses from their dermatology physician were extracted from the electronic medical record for analysis. The questions were input into ChatGPT (version 3.5) and the outputs extracted for analysis, with manual removal of verbiage pertaining to ChatGPT’s inability to provide medical advice. Ten blinded reviewers (seven physicians and three nonphysicians) rated and selected their preference in terms of ‘overall quality’, ‘readability’, ‘accuracy’, ‘thoroughness’ and ‘level of empathy’ of the physician- and ChatGPT-generated responses.

Results

Thirty-one messages and responses were analysed. Physician-generated responses were vastly preferred over the ChatGPT ­responses by the physician and nonphysician reviewers and received significantly higher ratings for ‘readability’ and ‘level of empathy’.

Conclusions

The results of this study suggest that physician-generated responses to patients’ portal messages are still preferred over ChatGPT, but generative AI tools may be helpful in generating the first drafts of responses and providing information on education resources for patients.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Electronic Health RecordsDermatologyNatural Language ProcessingHumansArtificial IntelligencePatient PortalsPhysician-Patient RelationsPhysicians

Autor:innen und Institutionen

  • Department of Dermatology, University of Michigan, Ann Arbor, MI, USA.

Zitieren

APA

Reynolds, K., Nadelman, D., Durgin, J., Ansah-Addo, S., Cole, D., Fayne, R., Harrell, J., Ratycz, M., Runge, M., Shepard-Hayes, A., Wenzel, D., & Tejasvi, T. (2024). Comparing the quality of ChatGPT- and physician-generated responses to patients’ dermatology questions in the electronic medical record. Clinical and Experimental Dermatology. https://doi.org/10.1093/ced/llad456

Vancouver

Reynolds K, Nadelman D, Durgin J, Ansah-Addo S, Cole D, Fayne R, et al. Comparing the quality of ChatGPT- and physician-generated responses to patients’ dermatology questions in the electronic medical record. Clinical and Experimental Dermatology. 2024. doi:10.1093/ced/llad456

BibTeX
@article{Reynolds2024,
  author = {Reynolds, Kelly and Nadelman, Daniel and Durgin, Joseph and Ansah-Addo, Stephen and Cole, Daniel and Fayne, Rachel and Harrell, Jane and Ratycz, Madison and Runge, Mason and Shepard-Hayes, Amanda and Wenzel, Daniel and Tejasvi, Trilokraj},
  title = {Comparing the quality of ChatGPT- and physician-generated responses to patients’ dermatology questions in the electronic medical record},
  journal = {Clinical and Experimental Dermatology},
  year = {2024},
  doi = {10.1093/ced/llad456}
}
Kostenlos starten →