From knowledge to judgment: A three-year longitudinal analysis of artificial intelligence large language model performance on the Chinese national nurse licensing examination
PLOS One · 02.07.2026 · 21 (7) · e0353059
DOI: 10.1371/journal.pone.0353059 ↗Kurz gefasst
Fachartikel in der Zeitschrift PLOS One, online erschienen am 02.07.2026. Verfasst von Xinju Zhan und 3 weiteren Autor:innen. Beteiligte Einrichtung: Nursing Department, Sir Run Run Shaw Hospital, Zhejiang University School of Medicine, Hangzhou, China. Hauptthemen laut MEDLINE: Licensure, Nursing, Artificial Intelligence, Educational Measurement. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 45 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Zhejiang Provincial Natural Science Foundation of China, Medical Science and Technology Project of Zhejiang Province.
Abstract (Originalsprache)
Background
The rapid advancement of Large Language Models (LLMs) presents unprecedented opportunities for healthcare education and professional credentialing. However, comprehensive longitudinal analyses of their evolving capabilities in nursing contexts remain limited.
Objective
To conduct a three-year longitudinal performance analysis of major international and Chinese-native LLMs on the Chinese National Nurse Licensing Examination (NNLE) from July 2022 to June 2025, examining performance trajectories, comparative effectiveness, and domain-specific competencies.
Methods
We curated a comprehensive corpus of 9,800 multiple-choice questions from NNLE examinations (2022–2025) through validated educational resources. Fifteen leading LLMs were evaluated using standardized zero-shot prompting protocols, with temporal fidelity ensuring models were tested only on examinations administered after their release dates. Performance was measured as raw accuracy and benchmarked against the approximate 300-point passing threshold. Statistical analyses included trend analysis, comparative performance testing, and qualitative error categorization.
Results
LLM performance demonstrated a steep upward trajectory, with top-tier models achieving accuracy rates from 47.0% in 2022 to 78.8% in 2025. Chinese-native models consistently outperformed international counterparts. The mean Chinese-native advantage decreased from 6.1 percentage points in 2023 to 3.0 percentage points in 2025, while the top-model advantage remained present but non-monotonic, measuring 4.5, 3.0, and 3.8 percentage points in 2023, 2024, and 2025, respectively. Models exhibited superior performance in the knowledge-oriented Professional Practice section (81.6% average accuracy) versus the application-oriented Practical Skills section (70.9% average accuracy). Clinical reasoning failures, particularly in nursing intervention prioritization, constituted 43% of errors among top-performing models.
Conclusion
While state-of-the-art LLMs demonstrate substantial codified nursing knowledge sufficient to achieve approximate passing thresholds on professional licensing examinations, significant deficiencies in complex clinical judgment persist, defining the current boundary between artificial intelligence capabilities and human professional competence. Critically, examination performance should not be interpreted as evidence of clinical readiness or autonomous practice capability.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- Nursing Department, Sir Run Run Shaw Hospital, Zhejiang University School of Medicine, Hangzhou, China.
- Department of General Surgery, Sir Run Run Shaw Hospital, Zhejiang University School of Medicine, Hangzhou, China.
ORCID-Profile: Jionghuang Chen
Förderung
- Zhejiang Provincial Natural Science Foundation of China
- Medical Science and Technology Project of Zhejiang Province
Zitieren
APA
Zhan, X., Yu, W., Cai, J., & Chen, J. (2026). From knowledge to judgment: A three-year longitudinal analysis of artificial intelligence large language model performance on the Chinese national nurse licensing examination. PLOS One, 21(7), e0353059. https://doi.org/10.1371/journal.pone.0353059
Vancouver
Zhan X, Yu W, Cai J, Chen J. From knowledge to judgment: A three-year longitudinal analysis of artificial intelligence large language model performance on the Chinese national nurse licensing examination. PLOS One. 2026;21(7):e0353059. doi:10.1371/journal.pone.0353059
BibTeX
@article{Zhan2026,
author = {Zhan, Xinju and Yu, Weihua and Cai, Jianshu and Chen, Jionghuang},
title = {From knowledge to judgment: A three-year longitudinal analysis of artificial intelligence large language model performance on the Chinese national nurse licensing examination},
journal = {PLOS One},
year = {2026},
volume = {21},
number = {7},
pages = {e0353059},
doi = {10.1371/journal.pone.0353059}
}
EUVIMED