EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC 3.0

Evaluating the Applicability of Advanced Large Language Models in Laboratory Medicine Test Questions: A Comparative Performance Study

Wenzheng Han, Wenkai Zhu, Gang Feng iD, Yankang Wang, Guang Chen, Huan Zhou, Bin Quan, Qiwen Wu, Jianghua Yang iD, Kai Jin, Shaoneng Tao, Xiaoning Li und 1 weitere

Advances in Medical Education and Practice · 2026-02 · Volume 17 · 1-13

DOI: 10.2147/amep.s576651 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Advances in Medical Education and Practice, 2026-02. Verfasst von Wenzheng Han und 12 weiteren Autor:innen. Beteiligte Einrichtung: Department of Clinical Laboratory, The First Affiliated Hospital, Wannan Medical College, Wuhu, Anhui, People's Republic of China. Die Arbeit nennt 21 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 3.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

While large language models (LLMs) show promise in medical education, their comprehensive performance in specialized domains like medical laboratory science remains inadequately assessed.

Purpose

This study aimed to evaluate advanced LLMs on medical laboratory questions, assessing accuracy, natural language generation (NLG) quality, reasoning performance, and efficiency.

Methods

We conducted a multi-faceted evaluation of three advanced LLMs (DeepSeek-R1, Gemini-2.5 Pro, GPT-5), benchmarking them against medical laboratory scientists and earlier ChatGPT versions. The evaluation utilized 493 questions sourced from the internal Medical Laboratory Test Bank of Wannan Medical College. These questions comprised both knowledge-based and reasoning-based single- and multiple-choice types (SCQs and MCQs). Performance was measured by accuracy, Macro-F1, response time, NLG scores (ROUGE-L, METEOR), and structured logical reasoning assessment. Appropriate statistical tests (including χ2, Wilcoxon, ANOVA, and non-parametric alternatives) with post-hoc corrections were applied to determine significance.

Results

DeepSeek-R1's accuracy on total questions was 78.3%, nearing the 79.3% of the higher-performing senior expert. Notably, it excelled at complex reasoning-based MCQ, demonstrating an advantage over senior experts with an accuracy of 64.4%, compared to 58.7% (SMLS-1) and 56.7% (SMLS-2). While ChatGPT-5 was the fastest model, DeepSeek-R1 exhibited intermediate efficiency, aligning with human experts on SCQ but requiring more time for MCQ. In terms of NLG, DeepSeek-R1 consistently achieved the highest scores, with ROUGE-L scores of 0.36 ± 0.14 (Total Q), 0.33 ± 0.15 (SCQ), and 0.38 ± 0.13 (MCQ), and METEOR scores of 0.53 ± 0.19 (Total Q), 0.40 ± 0.17 (SCQ), and 0.63 ± 0.14 (MCQ). Furthermore, it significantly outperformed all other LLMs in logical reasoning comprehensiveness. A critical strength was its consistent integration of key negative findings, vital for diagnosis.

Conclusion

DeepSeek-R1 approaches or even surpasses senior expert performance in certain tasks, showing strong potential as an effective tool for education and assessment despite slower processing times.

Schlagwörter

Stichwörter der Autor:innen

complex reasoningdeepseek-R1large language modelsmedical laboratory sciencenatural language generation

Autor:innen und Institutionen

  • Department of Clinical Laboratory, The First Affiliated Hospital, Wannan Medical College, Wuhu, Anhui, People's Republic of China.
  • Department of Pediatrics, The First Affiliated Hospital, Wannan Medical College, Wuhu, Anhui, People's Republic of China.
  • School of Laboratory Medicine, Wannan Medical College, Wuhu, Anhui, People's Republic of China.
  • Department of Infectious Diseases, The First Affiliated Hospital, Wannan Medical College, Wuhu, Anhui, People's Republic of China.
  • Department of Eye Center, The Second Affiliated Hospital, School of Medicine, Zhejiang University, Hangzhou, Zhejiang, People's Republic of China.
  • Department of Nuclear Medicine, The First Affiliated Hospital, Wannan Medical College, Wuhu, Anhui, People's Republic of China.

ORCID-Profile: Gang Feng, Jianghua Yang

Zitieren

APA

Han, W., Zhu, W., Feng, G., Wang, Y., Chen, G., Zhou, H., Quan, B., Wu, Q., Yang, J., Jin, K., Tao, S., Li, X., & Chen, Q. (2026). Evaluating the Applicability of Advanced Large Language Models in Laboratory Medicine Test Questions: A Comparative Performance Study. Advances in Medical Education and Practice, Volume 17, 1-13. https://doi.org/10.2147/amep.s576651

Vancouver

Han W, Zhu W, Feng G, Wang Y, Chen G, Zhou H, et al. Evaluating the Applicability of Advanced Large Language Models in Laboratory Medicine Test Questions: A Comparative Performance Study. Advances in Medical Education and Practice. 2026;Volume 17:1-13. doi:10.2147/amep.s576651

BibTeX
@article{Han2026,
  author = {Han, Wenzheng and Zhu, Wenkai and Feng, Gang and Wang, Yankang and Chen, Guang and Zhou, Huan and Quan, Bin and Wu, Qiwen and Yang, Jianghua and Jin, Kai and Tao, Shaoneng and Li, Xiaoning and Chen, Qing},
  title = {Evaluating the Applicability of Advanced Large Language Models in Laboratory Medicine Test Questions: A Comparative Performance Study},
  journal = {Advances in Medical Education and Practice},
  year = {2026},
  volume = {Volume 17},
  pages = {1--13},
  doi = {10.2147/amep.s576651}
}
Kostenlos starten →