EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Evaluation of the usability of ChatGPT-4 Pro and Gemini 2.5 Pro in patient education about brain tumors

Umut Ogün Mutlucan, Cihan Bedel, Fatih Selvi, Ökkeş Zortuk, Cezmi Çağrı Türk

Journal of Medical and Dental Investigations · 25.12.2025 · 6 · e250132

DOI: 10.5577/jomdi.e250132 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of Medical and Dental Investigations, online erschienen am 25.12.2025. Verfasst von Umut Ogün Mutlucan und 4 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Aim

The aim of this study is to determine the reliability of ChatGPT-4 Pro and Gemini 2.5 Pro chatbots through a systematic evaluation of the responses provided by neurosurgical specialists to patients' inquiries regarding brain tumors.

Methods

The present study was conducted using artificial intelligence programs, and there is no relationship between the authors and the AI companies and sites associated with the study. The final tally revealed that a total of 56 frequently asked questions were identified. The present study will examine the ChatGPT-4 Pro and Gemini 2.5 Pro. The responses furnished by both artificial intelligence models were produced in Turkish and subsequently assessed by two independent neurosurgeons. The evaluation of the responses was conducted by two independent evaluators, who assigned scores without the knowledge of each other's evaluations. In the event that two neurosurgeons assigned the same score to a given response, it was accepted as final. In instances of discordance, a collaborative discussion was initiated, culminating in the determination and documentation of a consensus score.

Results

The distribution of these questions is illustrated in Table 2. The mean GPT score for anatomy questions was 4.25 ± 0.88, while the mean GEMINI score was 4.50 ± 0.53 (p = 0.282). For inquiries pertaining to general questions, the mean GPT score was 4.43 ± 0.81, while the mean GEMINI score was 4.38 ± 0.81 (p = 0.500). Inquiries pertaining to prognostication and daily living activities revealed a mean GPT score of 5.00 ± 0.00, accompanied by a mean GEMINI score of 4.57 ± 0.78 (p = 0.100). In the treatment questions, the mean GPT score was 4.63 ± 0.67, while the mean GEMINI score was 4.36 ± 0.67 (p = 0.138). Figure 1 presents a comparative analysis categorized by inquiry group. A comparison of the mean scores revealed that GPT and GEMINI exhibited similar performance, with mean scores of 4.54 and 4.44, respectively.

Conclusion

The present study demonstrates that large language model (LLM) technologies, including ChatGPT-4 Pro and Gemini 2.5 Pro, exhibit considerable promise in the provision of information and guidance to patients. Furthermore, the investigation revealed that artificial intelligence models do not demonstrate a substantial degree of superiority in the education of patients regarding brain tumors.

Zitieren

APA

Mutlucan, U. O., Bedel, C., Selvi, F., Zortuk, Ö., & Türk, C. Ç. (2025). Evaluation of the usability of ChatGPT-4 Pro and Gemini 2.5 Pro in patient education about brain tumors. Journal of Medical and Dental Investigations, 6, e250132. https://doi.org/10.5577/jomdi.e250132

Vancouver

Mutlucan UO, Bedel C, Selvi F, Zortuk Ö, Türk CÇ. Evaluation of the usability of ChatGPT-4 Pro and Gemini 2.5 Pro in patient education about brain tumors. Journal of Medical and Dental Investigations. 2025;6:e250132. doi:10.5577/jomdi.e250132

BibTeX
@article{Mutlucan2025,
  author = {Mutlucan, Umut Ogün and Bedel, Cihan and Selvi, Fatih and Zortuk, Ökkeş and Türk, Cezmi Çağrı},
  title = {Evaluation of the usability of ChatGPT-4 Pro and Gemini 2.5 Pro in patient education about brain tumors},
  journal = {Journal of Medical and Dental Investigations},
  year = {2025},
  volume = {6},
  pages = {e250132},
  doi = {10.5577/jomdi.e250132}
}
Kostenlos starten →