EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Performance of ChatGPT-4o, Gemini 2.0 Pro, and DeepSeek-V3 in Patient-Facing Information on Chest Wall Deformities: A Comparative Evaluation of Accuracy, RELIABILITY, and Reproducibility

Deniz Oke iD, Ozge Gulsum Illeez, Esra Giray, Betül Çiftçi iD

Diagnostics · 15.02.2026 · 16 (4) · 589

DOI: 10.3390/diagnostics16040589 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Diagnostics, online erschienen am 15.02.2026. Verfasst von Deniz Oke und 3 weiteren Autor:innen. Beteiligte Einrichtung: Department of Physical Medicine and Rehabilitation, Health Sciences University Gaziosmanpasa Training and Research Hospital, Istanbul 34255, Turkey. Die Arbeit wurde bisher 6-mal zitiert (Crossref) und nennt 56 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Large language models (LLMs) such as DeepSeek-V3, Google Gemini 2.0 Pro, and ChatGPT-4o are increasingly used by patients seeking online medical information. However, their accuracy, reliability, and reproducibility in patient-facing content related to chest wall deformities (CWD) remain unclear. This study aimed to compare the performance of three contemporary LLMs in generating information on pectus excavatum, pectus carinatum, and related thoracic deformities.

Methods

Eighty patient-facing questions were developed across eight thematic domains and independently submitted to each model using newly created accounts over two consecutive days. Accuracy was assessed using a validated four-point rubric by blinded physiatrists, and reproducibility was evaluated using agreement metrics and weighted Cohen’s kappa.

Results

ChatGPT-4o achieved the highest overall accuracy (median score: 1.20), the greatest proportion of fully accurate responses, and the lowest hallucination rate (5.0%). Gemini showed intermediate accuracy, while DeepSeek-V3 demonstrated the lowest accuracy and highest hallucination rate (11.25%). Across all models, general-information and quality-of-life domains had the best performance, whereas treatment-related questions showed the most errors. Reproducibility was highest for ChatGPT-4o (weighted κ = almost perfect), followed by Gemini and DeepSeek-V3. Inter-rater reliability was substantial (Fleiss’ κ = 0.69).

Conclusions

Contemporary LLMs can generate largely accurate and reproducible patient-facing information on CWD, with ChatGPT-4o showing the strongest overall performance. This study provides the first domain-specific comparative evaluation of LLMs in CWD and integrates reproducibility analysis alongside accuracy and reliability assessment. While these tools may support patient education, treatment-related responses require caution, and LLMs should be used as adjuncts rather than substitutes for clinical counseling.

Schlagwörter

Stichwörter der Autor:innen

ChatGPT-4oDeepSeek-V3Google Gemini 2.0 Proaccuracyartificial intelligencechest wall deformityhallucinationspatient educationreproducibility

Autor:innen und Institutionen

  • Department of Physical Medicine and Rehabilitation, Health Sciences University Gaziosmanpasa Training and Research Hospital, Istanbul 34255, Turkey.
  • Department of Physical Medicine and Rehabilitation, Fatih Sultan Mehmet Research and Training Hospital, Istanbul 34015, Turkey.
  • Department of Physical Medicine and Rehabilitation, Kırklareli University Faculty of Medicine, Kırklareli 39060, Turkey.

ORCID-Profile: Deniz Oke, Betül Çiftçi

Zitieren

APA

Oke, D., Illeez, O. G., Giray, E., & Çiftçi, B. (2026). Performance of ChatGPT-4o, Gemini 2.0 Pro, and DeepSeek-V3 in Patient-Facing Information on Chest Wall Deformities: A Comparative Evaluation of Accuracy, RELIABILITY, and Reproducibility. Diagnostics, 16(4), 589. https://doi.org/10.3390/diagnostics16040589

Vancouver

Oke D, Illeez OG, Giray E, Çiftçi B. Performance of ChatGPT-4o, Gemini 2.0 Pro, and DeepSeek-V3 in Patient-Facing Information on Chest Wall Deformities: A Comparative Evaluation of Accuracy, RELIABILITY, and Reproducibility. Diagnostics. 2026;16(4):589. doi:10.3390/diagnostics16040589

BibTeX
@article{Oke2026,
  author = {Oke, Deniz and Illeez, Ozge Gulsum and Giray, Esra and Çiftçi, Betül},
  title = {Performance of ChatGPT-4o, Gemini 2.0 Pro, and DeepSeek-V3 in Patient-Facing Information on Chest Wall Deformities: A Comparative Evaluation of Accuracy, RELIABILITY, and Reproducibility},
  journal = {Diagnostics},
  year = {2026},
  volume = {16},
  number = {4},
  pages = {589},
  doi = {10.3390/diagnostics16040589}
}
Kostenlos starten →