EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Assessing the feasibility of ChatGPT-4o and Claude 3-Opus in thyroid nodule classification based on ultrasound images

Ziman Chen, Nonhlanhla Chambara, Chaoqun Wu, Xina Lo, Shirley Yuk Wah Liu, Simon Takadiyi Gunda, Xinyang Han, Jingguo Qu, Fei Chen, Michael Tin Cheung Ying

Endocrine · 11.10.2024 · 87 (3) · 1041-1049

DOI: 10.1007/s12020-024-04066-x ↗

Kurz gefasst

Fachartikel in der Zeitschrift Endocrine, online erschienen am 11.10.2024. Verfasst von Ziman Chen und 9 weiteren Autor:innen. Beteiligte Einrichtung: Department of Health Technology and Informatics, The Hong Kong Polytechnic University, Kowloon, Hong Kong, China. chenzm27@mail3.sysu.edu.cn. Hauptthemen laut MEDLINE: Thyroid Nodule. Die Arbeit wurde bisher 51-mal zitiert (Crossref) und nennt 31 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Hong Kong Polytechnic University.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Purpose

Large language models (LLMs) are pivotal in artificial intelligence, demonstrating advanced capabilities in natural language understanding and multimodal interactions, with significant potential in medical applications. This study explores the feasibility and efficacy of LLMs, specifically ChatGPT-4o and Claude 3-Opus, in classifying thyroid nodules using ultrasound images.

Methods

This study included 112 patients with a total of 116 thyroid nodules, comprising 75 benign and 41 malignant cases. Ultrasound images of these nodules were analyzed using ChatGPT-4o and Claude 3-Opus to diagnose the benign or malignant nature of the nodules. An independent evaluation by a junior radiologist was also conducted. Diagnostic performance was assessed using Cohen’s Kappa and receiver operating characteristic (ROC) curve analysis, referencing pathological diagnoses.

Results

ChatGPT-4o demonstrated poor agreement with pathological results ( Kappa = 0.116), while Claude 3-Opus showed even lower agreement ( Kappa = 0.034). The junior radiologist exhibited moderate agreement ( Kappa = 0.450). ChatGPT-4o achieved an area under the ROC curve (AUC) of 57.0% (95% CI: 48.6–65.5%), slightly outperforming Claude 3-Opus (AUC of 52.0%, 95% CI: 43.2–60.9%). In contrast, the junior radiologist achieved a significantly higher AUC of 72.4% (95% CI: 63.7–81.1%). The unnecessary biopsy rates were 41.4% for ChatGPT-4o, 43.1% for Claude 3-Opus, and 12.1% for the junior radiologist.

Conclusion

While LLMs such as ChatGPT-4o and Claude 3-Opus show promise for future applications in medical imaging, their current use in clinical diagnostics should be approached cautiously due to their limited accuracy.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Thyroid NoduleHumansUltrasonographyFemaleMaleFeasibility StudiesMiddle AgedAdultAgedArtificial IntelligenceGenerative Artificial Intelligence

Stichwörter der Autor:innen

Artificial intelligenceDiagnostic accuracyLarge language modelThyroid cancerUltrasound

Autor:innen und Institutionen

  • Department of Health Technology and Informatics, The Hong Kong Polytechnic University, Kowloon, Hong Kong, China. chenzm27@mail3.sysu.edu.cn.
  • School of Healthcare Sciences, Cardiff University, Cardiff, UK.
  • Department of Ultrasound, The First Affiliated Hospital of Wenzhou Medical University, Wenzhou, China.
  • Department of Surgery, North District Hospital, Sheung Shui, New Territories, Hong Kong, China.
  • Department of Surgery, The Chinese University of Hong Kong, Prince of Wales Hospital, Shatin, New Territories, Hong Kong, China.
  • Department of Health Technology and Informatics, The Hong Kong Polytechnic University, Kowloon, Hong Kong, China.

Förderung

  • Hong Kong Polytechnic University

Zitieren

APA

Chen, Z., Chambara, N., Wu, C., Lo, X., Liu, S. Y. W., Gunda, S. T., Han, X., Qu, J., Chen, F., & Ying, M. T. C. (2024). Assessing the feasibility of ChatGPT-4o and Claude 3-Opus in thyroid nodule classification based on ultrasound images. Endocrine, 87(3), 1041-1049. https://doi.org/10.1007/s12020-024-04066-x

Vancouver

Chen Z, Chambara N, Wu C, Lo X, Liu SYW, Gunda ST, et al. Assessing the feasibility of ChatGPT-4o and Claude 3-Opus in thyroid nodule classification based on ultrasound images. Endocrine. 2024;87(3):1041-1049. doi:10.1007/s12020-024-04066-x

BibTeX
@article{Chen2024,
  author = {Chen, Ziman and Chambara, Nonhlanhla and Wu, Chaoqun and Lo, Xina and Liu, Shirley Yuk Wah and Gunda, Simon Takadiyi and Han, Xinyang and Qu, Jingguo and Chen, Fei and Ying, Michael Tin Cheung},
  title = {Assessing the feasibility of ChatGPT-4o and Claude 3-Opus in thyroid nodule classification based on ultrasound images},
  journal = {Endocrine},
  year = {2024},
  volume = {87},
  number = {3},
  pages = {1041--1049},
  doi = {10.1007/s12020-024-04066-x}
}
Kostenlos starten →