EUVIMEDEUVIMED entdecken
VergleichsstudieOpen Access

Performance of Cloud-Hosted Large Vision-Language Models on the Japanese National Examination for Clinical Laboratory Technicians: Comparative Benchmarking Study

Kota Murakami iD, Ryosuke Matsuzawa iD, Yuya Tahara-Arai iD, Haruka Ozaki iD

JMIR Medical Education · 11.09.2026 · 12 · e84266-e84266

DOI: 10.2196/84266 ↗

Kurz gefasst

Vergleichsstudie in der Zeitschrift JMIR Medical Education, online erschienen am 11.09.2026. Verfasst von Kota Murakami und 3 weiteren Autor:innen. Beteiligte Einrichtung: Master's Program in Medical Sciences, Graduate School of Comprehensive Human Sciences, University of Tsukuba, Ibaraki, Japan. Hauptthemen laut MEDLINE: Benchmarking, Clinical Competence, Laboratories, Clinical, Large Language Models. Die Arbeit nennt 34 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Cloud-hosted large vision-language models (LVLMs) often outperform open-weight models on multimodal benchmarks, but their applicability to health care examinations that require both text and image reasoning remains unclear. Existing studies on Japan’s National Examination for Clinical Laboratory Technicians mainly used text-only settings and a limited set of models.

Objective

This study aimed to evaluate the accuracy and applicability of cloud-hosted LVLMs on this examination, quantify the contribution of images, examine alignment with human performance across medical subfields, and test generalizability on questions administered after the models’ knowledge cutoff dates. We aim to provide a standardized benchmark of baseline multimodal performance under deployment-realistic conditions without additional fine-tuning.

Methods

For this comparative benchmarking study, we built the Kensagishi question-answer dataset (Kensagishi QA), a benchmark of 800 questions from the examination (2020‐2023), including text-only and image-based questions, in a standardized structured JSON format, and strict scoring criteria (all correct options required). We tested GPT-5, GPT-4o, GPT-4o-mini, Gemini 1.5 Pro, Gemini 2.5 Pro, and Neva-22B using a uniform “numbers-only” response prompt; images were provided as Base64-encoded data when available. Accuracy was computed overall, by item type, and by medical subfield. Human reference was drawn from published examination results. Spearman correlation assessed model-human alignment. Generalization was evaluated on 200 questions from the 71st examination (2025), released after the models’ knowledge cutoffs.

Results

Top-performing models exceeded the 60% passing threshold. GPT-5 achieved 93.9% (95% CI 92.0%‐95.4%) accuracy with images and 88.2% (95% CI 85.7%‐90.2%) without images, Gemini 2.5 Pro 92.5% (95% CI 90.5%‐94.1%) and 86.8% (95% CI 84.2%‐88.9%), and GPT-4o 81.7% (95% CI 78.8%‐84.2%) and 79.8% (95% CI 76.8%‐82.4%), respectively. Providing images markedly improved accuracy for GPT-5, Gemini 2.5 Pro, and GPT-4o, whereas Neva-22B showed no improvement. Model-human rank correlations by medical subfield were moderate for GPT-4o (Spearman ρ=0.61; 95% CI –0.03 to 0.90; P =.06 with images) and statistically significant for Gemini 1.5 Pro without images (ρ=0.67; 95% CI 0.07‐0.91; P =.03). On 200 postknowledge cutoff questions from the 71st examination (2025), model accuracies were comparable to historical averages (eg, GPT-5 91.5%, 95% CI 86.8%‐94.6%; Gemini 2.5 Pro 90.7%, 95% CI 85.8%‐94.0%; GPT-4o 82.0%, 95% CI 76.1%‐86.7%), indicating no material performance degradation on unseen questions.

Conclusions

Cloud-hosted LVLMs show strong performance on this examination, and visual input is a key driver of accuracy for leading models. Models that better mirror human difficulty patterns (eg, GPT-4o) offer complementary insights for educational analytics, whereas the highest-accuracy models (eg, GPT-5 and Gemini 2.5 Pro) may be preferable when correctness is paramount. To our knowledge, this is the first systematic, multimodel evaluation isolating visual input that extends prior text-only, few-model studies and provides Kensagishi QA to guide educational deployment of medical AI.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

BenchmarkingClinical CompetenceLaboratories, ClinicalLarge Language ModelsHumansJapan

Stichwörter der Autor:innen

Japanese National Examination for Clinical Laboratory Technicianslarge vision-language modelsmedical AImodel performance evaluationmultimodal reasoning

Autor:innen und Institutionen

  • Master's Program in Medical Sciences, Graduate School of Comprehensive Human Sciences, University of Tsukuba, Ibaraki, Japan.
  • Bioinformatics Laboratory, Institute of Medicine, University of Tsukuba, Ibaraki, Japan.
  • Laboratory for AI Biology, RIKEN Center for Biosystems Dynamics Research, 6-7-1 Minatojima Minamimachi, Chuo-ku, Kobe, Hyogo, 650-0047, Japan, 81 78-306-0111.
  • Doctoral Program in Medical Sciences, Graduate School of Comprehensive Human Sciences, University of Tsukuba, Ibaraki, Japan.
  • Ph.D. Program in Humanics, School of Integrative and Global Majors, University of Tsukuba, Ibaraki, Japan.

ORCID-Profile: Kota Murakami, Ryosuke Matsuzawa, Yuya Tahara-Arai, Haruka Ozaki

Zitieren

APA

Murakami, K., Matsuzawa, R., Tahara-Arai, Y., & Ozaki, H. (2026). Performance of Cloud-Hosted Large Vision-Language Models on the Japanese National Examination for Clinical Laboratory Technicians: Comparative Benchmarking Study. JMIR Medical Education, 12, e84266-e84266. https://doi.org/10.2196/84266

Vancouver

Murakami K, Matsuzawa R, Tahara-Arai Y, Ozaki H. Performance of Cloud-Hosted Large Vision-Language Models on the Japanese National Examination for Clinical Laboratory Technicians: Comparative Benchmarking Study. JMIR Medical Education. 2026;12:e84266-e84266. doi:10.2196/84266

BibTeX
@article{Murakami2026,
  author = {Murakami, Kota and Matsuzawa, Ryosuke and Tahara-Arai, Yuya and Ozaki, Haruka},
  title = {Performance of Cloud-Hosted Large Vision-Language Models on the Japanese National Examination for Clinical Laboratory Technicians: Comparative Benchmarking Study},
  journal = {JMIR Medical Education},
  year = {2026},
  volume = {12},
  pages = {e84266--e84266},
  doi = {10.2196/84266}
}
Kostenlos starten →