EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC-ND 4.0

Cross-Lingual Evaluation of Diagnostic Accuracy Across Patient Sex in LLMs

Hajer Janabi, Anum Khan, Chaturya Pasupuleti, Siddhi Pareek, Raj Davuluri, Ayush Davuluri, Rishit Lalchandani, Sai Prabhav Vankamamidi

International Conference on AI Research · 18.11.2026 · 7 (1) · 143-146

DOI: 10.34190/icair.7.1.5594 ↗

Kurz gefasst

Fachartikel in der Zeitschrift International Conference on AI Research, online erschienen am 18.11.2026. Verfasst von Hajer Janabi und 7 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Large language models (LLMs) are increasingly used to answer health questions across languages, making equitable diagnostic performance an important safety concern. This study examined whether patient sex and input language jointly affected diagnostic accuracy in GPT-4o mini, Claude Sonnet 4.6, and Gemini Flash-Lite 3.1. A total of 669 synthetic DDXPlus profiles (340 female and 329 male) were evaluated in English, Spanish, French, Hindi, Arabic, and Mandarin using an automated Python workflow. The primary outcome was exact-match diagnostic accuracy; technical response success and model-reported confidence were secondary outcomes. Across the five non-English languages, Claude achieved the highest diagnostic accuracy (29.9%), followed by Gemini (27.1%) and GPT-4o mini (22.7%). Contrary to the hypothesis, female profiles achieved higher accuracy than male profiles across all models: 32.6% versus 27.2% for Claude, 30.0% versus 24.1% for Gemini, and 26.6% versus 18.8% for GPT-4o mini. Accuracy also varied substantially across languages. These findings did not support the predicted intersectional disadvantage for female profiles, but they demonstrate that multilingual medical LLM performance depends on both the model and evaluation context. Rigorous cross-lingual and demographic validation is needed before these systems are used for clinical decision support.

Zitieren

APA

Janabi, H., Khan, A., Pasupuleti, C., Pareek, S., Davuluri, R., Davuluri, A., Lalchandani, R., & Vankamamidi, S. P. (2026). Cross-Lingual Evaluation of Diagnostic Accuracy Across Patient Sex in LLMs. International Conference on AI Research, 7(1), 143-146. https://doi.org/10.34190/icair.7.1.5594

Vancouver

Janabi H, Khan A, Pasupuleti C, Pareek S, Davuluri R, Davuluri A, et al. Cross-Lingual Evaluation of Diagnostic Accuracy Across Patient Sex in LLMs. International Conference on AI Research. 2026;7(1):143-146. doi:10.34190/icair.7.1.5594

BibTeX
@article{Janabi2026,
  author = {Janabi, Hajer and Khan, Anum and Pasupuleti, Chaturya and Pareek, Siddhi and Davuluri, Raj and Davuluri, Ayush and Lalchandani, Rishit and Vankamamidi, Sai Prabhav},
  title = {Cross-Lingual Evaluation of Diagnostic Accuracy Across Patient Sex in LLMs},
  journal = {International Conference on AI Research},
  year = {2026},
  volume = {7},
  number = {1},
  pages = {143--146},
  doi = {10.34190/icair.7.1.5594}
}
Kostenlos starten →