ChatGPT vs Medical Professional: Analyzing Responses to Laboratory Medicine Questions on Social Media
Clinical Chemistry · 03.09.2024 · 70 (9) · 1122-1139
DOI: 10.1093/clinchem/hvae093 ↗Kurz gefasst
Fachartikel in der Zeitschrift Clinical Chemistry, online erschienen am 16.07.2024. Verfasst von Mark R Girton und 4 weiteren Autor:innen. Beteiligte Einrichtung: Department of Pathology, University of Michigan, Ann Arbor, MI, United States. Hauptthemen laut MEDLINE: Social Media. Die Arbeit wurde bisher 33-mal zitiert (Crossref) und nennt 27 Literaturangaben.
Abstract (Originalsprache)
Background
The integration of ChatGPT, a large language model (LLM) developed by OpenAI, into healthcare has sparked significant interest due to its potential to enhance patient care and medical education. With the increasing trend of patients accessing laboratory results online, there is a pressing need to evaluate the effectiveness of ChatGPT in providing accurate laboratory medicine information. Our study evaluates ChatGPT's effectiveness in addressing patient questions in this area, comparing its performance with that of medical professionals on social media.
Methods
This study sourced patient questions and medical professional responses from Reddit and Quora, comparing them with responses generated by ChatGPT versions 3.5 and 4.0. Experienced laboratory medicine professionals evaluated the responses for quality and preference. Evaluation results were further analyzed using R software.
Results
The study analyzed 49 questions, with evaluators reviewing responses from both medical professionals and ChatGPT. ChatGPT's responses were preferred by 75.9% of evaluators and generally received higher ratings for quality. They were noted for their comprehensive and accurate information, whereas responses from medical professionals were valued for their conciseness. The interrater agreement was fair, indicating some subjectivity but a consistent preference for ChatGPT's detailed responses.
Conclusions
ChatGPT demonstrates potential as an effective tool for addressing queries in laboratory medicine, often surpassing medical professionals in response quality. These results support the need for further research to confirm ChatGPT's utility and explore its integration into healthcare settings.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- Department of Pathology, University of Michigan, Ann Arbor, MI, United States.
- Department of Laboratory Medicine and Pathology, University of Washington, Seattle, WA, United States.
- Department of Pathology, Women and Infants Hospital, Brown University, Providence, RI, United States.
- Department of Pathology, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA, United States.
ORCID-Profile: Mark R Girton, Dina N Greene, Geralyn Messerlian, Min Yu
Zitieren
APA
Girton, M. R., Greene, D. N., Messerlian, G., Keren, D. F., & Yu, M. (2024). ChatGPT vs Medical Professional: Analyzing Responses to Laboratory Medicine Questions on Social Media. Clinical Chemistry, 70(9), 1122-1139. https://doi.org/10.1093/clinchem/hvae093
Vancouver
Girton MR, Greene DN, Messerlian G, Keren DF, Yu M. ChatGPT vs Medical Professional: Analyzing Responses to Laboratory Medicine Questions on Social Media. Clinical Chemistry. 2024;70(9):1122-1139. doi:10.1093/clinchem/hvae093
BibTeX
@article{Girton2024,
author = {Girton, Mark R and Greene, Dina N and Messerlian, Geralyn and Keren, David F and Yu, Min},
title = {ChatGPT vs Medical Professional: Analyzing Responses to Laboratory Medicine Questions on Social Media},
journal = {Clinical Chemistry},
year = {2024},
volume = {70},
number = {9},
pages = {1122--1139},
doi = {10.1093/clinchem/hvae093}
}
EUVIMED