EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Bilingual performance of ChatGPT, Gemini, and DeepSeek in asthma, allergy, and respiratory infection queries

Mohammed Sallam, Adrian Stanley, Johan Snygg, Hasanain Al-Shakerchi, Omar Al Atragchi, Rania Abusamra, Malik Sallam

Recent Progress in Sciences · 23.01.2026 · 3 (1) · 001

DOI: 10.70462/rps.2026.3.001 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Recent Progress in Sciences, online erschienen am 23.01.2026. Verfasst von Mohammed Sallam und 6 weiteren Autor:innen. Die Arbeit wurde bisher 3-mal zitiert (Crossref) und nennt 129 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Generative artificial intelligence (genAI) models are rapidly being adopted for health information delivery. Nevertheless, systematic cross-linguistic evaluations of their clinical reliability—particularly in high-burden conditions such as asthma, allergy, and respiratory tract infections (RTIs)—remain limited. The aim of this study was to compare the English and Arabic performance of ChatGPT‑4o, Gemini, and DeepSeek in responding to common asthma, allergy, and RTI queries using a validated clinical assessment framework. A bilingual evaluation was conducted using 30 frequently asked questions (FAQs) related to asthma, allergy, and RTIs. Each question was submitted in English and Arabic to ChatGPT‑4o, Gemini, and DeepSeek. Responses were evaluated independently by three bilingual clinical experts using the CLEAR framework for Completeness, Accuracy, and Relevance of the generated content. Inter-rater reliability was assessed using intraclass correlation coefficients (ICCs). Language and model comparisons were analyzed using non-parametric Kruskal-Wallis and Mann-Whitney U tests. The study followed the METRICS reporting guideline for genAI in healthcare. ChatGPT‑4o consistently outperformed Gemini and DeepSeek across all CLEAR dimensions and the two languages. In English, the mean CLEAR scores were: ChatGPT‑4o: 3.90, Gemini: 2.50, DeepSeek: 2.09. In Arabic, ChatGPT‑4o again scored highest (3.63), followed by Gemini (2.38) and DeepSeek (1.84). All inter-model differences were statistically significant (p < 0.001). Inter-rater reliability was excellent across dimensions: ICC for completeness = 0.858, accuracy = 0.917, relevance = 0.950 (all p < 0.001), confirming strong consistency and validity in scoring. Within each genAI model, English outputs significantly outperformed Arabic in completeness, accuracy, relevance, and the overall CLEAR score. Domain-wise, asthma queries achieved the highest performance across models and languages, while allergy queries showed the lowest accuracy. ChatGPT‑4o demonstrated superior bilingual performance, while Gemini and DeepSeek exhibited significant limitations, particularly in Arabic. These findings highlight persistent language-based disparities in genAI health outputs. Rigorous cross-linguistic evaluation and domain-specific fine-tuning are essential to ensure safe and equitable deployment of genAI tools in global health communication.

Zitieren

APA

Sallam, M., Stanley, A., Snygg, J., Al-Shakerchi, H., Atragchi, O. A., Abusamra, R., & Sallam, M. (2026). Bilingual performance of ChatGPT, Gemini, and DeepSeek in asthma, allergy, and respiratory infection queries. Recent Progress in Sciences, 3(1), 001. https://doi.org/10.70462/rps.2026.3.001

Vancouver

Sallam M, Stanley A, Snygg J, Al-Shakerchi H, Atragchi OA, Abusamra R, et al. Bilingual performance of ChatGPT, Gemini, and DeepSeek in asthma, allergy, and respiratory infection queries. Recent Progress in Sciences. 2026;3(1):001. doi:10.70462/rps.2026.3.001

BibTeX
@article{Sallam2026,
  author = {Sallam, Mohammed and Stanley, Adrian and Snygg, Johan and Al-Shakerchi, Hasanain and Atragchi, Omar Al and Abusamra, Rania and Sallam, Malik},
  title = {Bilingual performance of ChatGPT, Gemini, and DeepSeek in asthma, allergy, and respiratory infection queries},
  journal = {Recent Progress in Sciences},
  year = {2026},
  volume = {3},
  number = {1},
  pages = {001},
  doi = {10.70462/rps.2026.3.001}
}
Kostenlos starten →