Comparative analysis of ChatGPT and Bard in answering pathology examination questions requiring image interpretation
American Journal of Clinical Pathology · 03.09.2024 · 162 (3) · 252-260
DOI: 10.1093/ajcp/aqae036 ↗Kurz gefasst
Vergleichsstudie in der Zeitschrift American Journal of Clinical Pathology, online erschienen am 15.04.2024. Verfasst von Sompon Apornvirat und 2 weiteren Autor:innen. Beteiligte Einrichtung: Division of Pathology, Chulabhorn International College of Medicine, Thammasat University, Pathum Thani, Thailand. Hauptthemen laut MEDLINE: Artificial Intelligence. Die Arbeit wurde bisher 23-mal zitiert (Crossref) und nennt 25 Literaturangaben.
Abstract (Originalsprache)
Objectives
To evaluate the accuracy of ChatGPT and Bard in answering pathology examination questions requiring image interpretation.
Methods
The study evaluated ChatGPT-4 and Bard’s performance using 86 multiple-choice questions, with 17 (19.8%) focusing on general pathology and 69 (80.2%) on systemic pathology. Of these, 62 (72.1%) included microscopic images, and 57 (66.3%) were first-order questions focusing on diagnosing the disease. The authors presented these artificial intelligence (AI) tools with questions, both with and without clinical contexts, and assessed their answers against a reference standard set by pathologists.
Results
ChatGPT-4 achieved a 100% (n = 86) accuracy rate in questions with clinical context, surpassing Bard’s 87.2% (n = 75). Without context, the accuracy of both AI tools declined significantly, with ChatGPT-4 at 52.3% (n = 45) and Bard at 38.4% (n = 33). ChatGPT-4 consistently outperformed Bard across various categories, particularly in systemic pathology and first-order questions. A notable issue identified was Bard’s tendency to “hallucinate” or provide plausible but incorrect answers, especially without clinical context.
Conclusions
This study demonstrated the potential of ChatGPT and Bard in pathology education, stressing the importance of clinical context for accurate AI interpretations of pathology images. It underlined the need for careful AI integration in medical education.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Division of Pathology, Chulabhorn International College of Medicine, Thammasat University, Pathum Thani, Thailand.
- Division of Pathology, Thammasat University Hospital, Pathum Thani, Thailand.
- Dr Pong Clinic, Bangkok, Thailand.
ORCID-Profile: Sompon Apornvirat, Chutimon Namboonlue, Thiyaphat Laohawetwanit
Zitieren
APA
Apornvirat, S., Namboonlue, C., & Laohawetwanit, T. (2024). Comparative analysis of ChatGPT and Bard in answering pathology examination questions requiring image interpretation. American Journal of Clinical Pathology, 162(3), 252-260. https://doi.org/10.1093/ajcp/aqae036
Vancouver
Apornvirat S, Namboonlue C, Laohawetwanit T. Comparative analysis of ChatGPT and Bard in answering pathology examination questions requiring image interpretation. American Journal of Clinical Pathology. 2024;162(3):252-260. doi:10.1093/ajcp/aqae036
BibTeX
@article{Apornvirat2024,
author = {Apornvirat, Sompon and Namboonlue, Chutimon and Laohawetwanit, Thiyaphat},
title = {Comparative analysis of ChatGPT and Bard in answering pathology examination questions requiring image interpretation},
journal = {American Journal of Clinical Pathology},
year = {2024},
volume = {162},
number = {3},
pages = {252--260},
doi = {10.1093/ajcp/aqae036}
}
EUVIMED