Comparative analysis of large language models' performance in breast ımaging
Turkish Journal of Clinics and Laboratory · 31.12.2024 · 15 (4) · 542-546
DOI: 10.18663/tjcl.1561361 ↗Kurz gefasst
Fachartikel in der Zeitschrift Turkish Journal of Clinics and Laboratory, online erschienen am 31.12.2024. Verfasst von Muhammed Said Beşler. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 14 Literaturangaben.
Abstract (Originalsprache)
Aim
To evaluate the performance of the flagship models, OpenAI's GPT-4o and Anthropic's Claude 3.5 Sonnet, in breast imaging cases. Material and Methods: The dataset consisted of cases from the publicly available Case of the Month archive by the Society of Breast Imaging. Questions were classified as text-based or containing images from mammography, ultrasound, magnetic resonance imaging, or hybrid imaging. The accuracy rates of GPT-4o and Claude 3.5 Sonnet were compared using the Mann-Whitney U test.
Results
Of the total 94 questions, 61.7% were image-based. The overall accuracy rate of GPT-4o was higher than that of Claude 3.5 Sonnet (75.4% vs. 67.7%, p=0.432). GPT-4o achieved higher scores on questions based on ultrasound and hybrid imaging, while Claude 3.5 Sonnet performed better on mammography-based questions. In tumor group cases, both models reached higher accuracy rates compared to the non-tumor group (both, p>0.05). The models' performance in breast imaging cases overall exceeded 75%, ranging between 64-83% for questions involving different imaging modalities.
Conclusion
In breast imaging cases, although GPT-4o generally achieved higher accuracy rates than Claude 3.5 Sonnet in image-based and other types of questions, their performances were comparable.
Zitieren
APA
Beşler, M. S. (2024). Comparative analysis of large language models' performance in breast ımaging. Turkish Journal of Clinics and Laboratory, 15(4), 542-546. https://doi.org/10.18663/tjcl.1561361
Vancouver
Beşler MS. Comparative analysis of large language models' performance in breast ımaging. Turkish Journal of Clinics and Laboratory. 2024;15(4):542-546. doi:10.18663/tjcl.1561361
BibTeX
@article{Beler2024,
author = {Beşler, Muhammed Said},
title = {Comparative analysis of large language models' performance in breast ımaging},
journal = {Turkish Journal of Clinics and Laboratory},
year = {2024},
volume = {15},
number = {4},
pages = {542--546},
doi = {10.18663/tjcl.1561361}
}
EUVIMED