EUVIMEDObjevte EUVIMED
QuerschnittstudieOpen access

Diagnostic Accuracy and Fine-Grained Image-Interpretation Performance of Multimodal Large Language Models on Bilingual Radiology Questions: Cross-Sectional Benchmark Study

Yijie Liu, Xin Zhang, Jia Xu, Manxin Huang, Yinting Xiong, Wenqian Wu

Journal of Medical Internet Research · 2026-11-11 · 28 · e98849-e98849

DOI: 10.2196/98849 ↗

At a glance

Querschnittstudie in Journal of Medical Internet Research, published online on 2026-11-11. Written by Yijie Liu and 5 co-authors. Institution: Union Hospital, Department of Ultrasound Medicine, Tongji Medical College, Huazhong University of Science and Technology, 1277 Jiefang Avenue, Wuhan,…. Major MEDLINE topics: Multilingualism, Radiology, Image Interpretation, Computer-Assisted. 27 references. The full text is freely available in PubMed Central.

AbstraktSubjectsBibliografické údajeCiteRelated studies

Abstract (original language)

Background

Multimodal large language models (LLMs) are rapidly expanding in radiology and digital health; however, their diagnostic performance across diverse clinical scenarios and fine-grained visual reasoning tasks requires systematic evaluation.

Objective

The study aims to evaluate the diagnostic accuracy and fine-grained visual reasoning performance of 5 multimodal LLMs with bilingual medical imaging questions.

Methods

This cross-sectional benchmark study evaluated bilingual medical imaging questions using zero-shot prompting on 5 LLMs: GPT-5.2, Gemini 3 Pro, Doubao, Yuanbao, and Llama 4 Maverick. Subgroup analyses by language (English/Chinese), question type (image/text), imaging modality (computed tomography, radiography, magnetic resonance imaging, ultrasound, and nuclear medicine), and anatomical system (neurologic, respiratory, cardiovascular, digestive, genitourinary, and musculoskeletal) evaluated intermodel accuracy and intramodel consistency. Image-based visual reasoning performance was assessed across modality, anatomy, lesion detection, and localization.

Results

A total of 298 questions (198 image-based, 100 text-based) were evaluated. Gemini 3 Pro achieved the highest overall accuracy (n=268, 89.9%, 95% CI 86.0%‐92.9%), significantly outperforming Doubao (n=235, 78.9%; P

Subjects

MeSH terms (MEDLINE)

MultilingualismRadiologyImage Interpretation, Computer-AssistedLarge Language ModelsCross-Sectional StudiesHumansBenchmarking

Author keywords

AIbenchmarkinglarge language modelsmedical imagingmultiple-choice questionsradiology

Authors and institutions

  • Union Hospital, Department of Ultrasound Medicine, Tongji Medical College, Huazhong University of Science and Technology, 1277 Jiefang Avenue, Wuhan, 430022, China., Wuhan, Hubei, China, 86 27-85726430, 86 27-85726430.
  • Hubei Province Clinical Research Center for Medical Imaging, Wuhan, Hubei, China.
  • Hubei Province Key Laboratory of Molecular Imaging, Wuhan, Hubei, China.

Cite

APA

Liu, Y., Zhang, X., Xu, J., Huang, M., Xiong, Y., & Wu, W. (2026). Diagnostic Accuracy and Fine-Grained Image-Interpretation Performance of Multimodal Large Language Models on Bilingual Radiology Questions: Cross-Sectional Benchmark Study. Journal of Medical Internet Research, 28, e98849-e98849. https://doi.org/10.2196/98849

Vancouver

Liu Y, Zhang X, Xu J, Huang M, Xiong Y, Wu W. Diagnostic Accuracy and Fine-Grained Image-Interpretation Performance of Multimodal Large Language Models on Bilingual Radiology Questions: Cross-Sectional Benchmark Study. Journal of Medical Internet Research. 2026;28:e98849-e98849. doi:10.2196/98849

BibTeX
@article{Liu2026,
  author = {Liu, Yijie and Zhang, Xin and Xu, Jia and Huang, Manxin and Xiong, Yinting and Wu, Wenqian},
  title = {Diagnostic Accuracy and Fine-Grained Image-Interpretation Performance of Multimodal Large Language Models on Bilingual Radiology Questions: Cross-Sectional Benchmark Study},
  journal = {Journal of Medical Internet Research},
  year = {2026},
  volume = {28},
  pages = {e98849--e98849},
  doi = {10.2196/98849}
}
Start for free →