Assessment of a large language model’s utility in helping pathology professionals answer general knowledge pathology questions
American Journal of Clinical Pathology · 2024
DOI: 10.1093/ajcp/aqad106 ↗Kurz gefasst
Fachartikel in der Zeitschrift American Journal of Clinical Pathology, 2024. Verfasst von Robert C Benirschke und 5 weiteren Autor:innen. Beteiligte Einrichtung: Department of Pathology and Laboratory Medicine, NorthShore University HealthSystem, Evanston, IL, US. Hauptthemen laut MEDLINE: Pathologists, Pathology, Clinical. Die Arbeit wurde bisher 19-mal zitiert (Crossref) und nennt 21 Literaturangaben.
Abstract (Originalsprache)
Objectives
To assess the utility and performance of the large language model ChatGPT 4.0 regarding accuracy, completeness, and its potential as a time-saving tool for pathologists and laboratory directors.
Methods
A deidentified database of questions previously sent to pathology residents from health care providers was used as a source of general knowledge–type pathology questions. These questions were submitted to the large language model and the responses graded by subject matter experts in different pathology subspecialties. The grading criteria assessed accuracy, completeness, and the potential time savings for helping the pathologist craft the response.
Results
Overall, respondents thought that most of the answers would take less than 5 minutes of additional work to be used (85%). Accuracy and completeness for the 61 questions was high, with 98% of responses being at least “completely or mostly accurate” and 82% of responses “containing all relevant information.” Of the respondents, 97% stated that the response would have “zero or near-zero potential for medical harm,” and all thought the tool had potential to save time in constructing answers to health care providers’ queries. Performance was similar in both Anatomic Pathology (AP) and Clinical Pathology (CP), with the only exception being some relevant information was excluded in 46% of AP answers vs only 10% in CP (P < .01).
Conclusions
ChatGPT version 4.0 gave responses that were predominantly accurate and complete for general knowledge–type pathology questions. With further research and when reviewed by a pathologist or laboratorian, this could facilitate its use as a pathologist’s aid in answering questions from health care providers.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- Department of Pathology and Laboratory Medicine, NorthShore University HealthSystem, Evanston, IL, US.
- Department of Pathology, University of Chicago Pritzker School of Medicine, Chicago, IL, US.
ORCID-Profile: Joshua Wodskow
Zitieren
APA
Benirschke, R. C., Wodskow, J., Prasai, K., Freeman, A., Lee, J. M., & Groth, J. (2024). Assessment of a large language model’s utility in helping pathology professionals answer general knowledge pathology questions. American Journal of Clinical Pathology. https://doi.org/10.1093/ajcp/aqad106
Vancouver
Benirschke RC, Wodskow J, Prasai K, Freeman A, Lee JM, Groth J. Assessment of a large language model’s utility in helping pathology professionals answer general knowledge pathology questions. American Journal of Clinical Pathology. 2024. doi:10.1093/ajcp/aqad106
BibTeX
@article{Benirschke2024,
author = {Benirschke, Robert C and Wodskow, Joshua and Prasai, Kritika and Freeman, Andrew and Lee, John M and Groth, John},
title = {Assessment of a large language model’s utility in helping pathology professionals answer general knowledge pathology questions},
journal = {American Journal of Clinical Pathology},
year = {2024},
doi = {10.1093/ajcp/aqad106}
}
EUVIMED