Outcomes, Evaluation Metrics, and Measurement Tools for Large Language Model Applications Research in Nursing: A Scoping Review
Outcomes, Evaluation Metrics und Measurement Tools für Forschungsanwendungen von Large Language Models in der Pflege: Scoping Review
International Nursing Review · 2026-09 · 73 (3)
DOI: 10.1111/inr.70217 ↗Auf einen Blick
Die Scoping Review identifizierte 50 Studien zu LLM-Anwendungen in der Pflege. Die berichteten Outcomes und Messinstrumente waren heterogen und unzureichend standardisiert.
Fragestellung
Welche Outcomes, Evaluationsmetriken und Messinstrumente wurden in Studien zu LLM-Anwendungen in der Pflege verwendet?
Methode
Scoping Review nach Joanna Briggs Institute und PRISMA-ScR; 8 englische und chinesische Datenbanken; Einordnung nach COMET und TRIPOD-LLM.
Ergebnisse
50 Studien erfüllten die Einschlusskriterien. Es wurden 56 verschiedene evaluative Elemente extrahiert und drei COMET-Domänen zugeordnet: objektive Effizienzindikatoren, intermediäre Effekte und subjektive Erfahrung.
Einschränkungen
Heterogene und inkonsistent berichtete Outcomes, Metriken und Messinstrumente; geringe Standardisierung.
Schlussfolgerung der Autor:innen
Die Übersichtsarbeit unterstützt eine konsistentere Berichterstattung und Strukturierung von LLM-Anwendungen in Pflegeforschung und -praxis.
Kernaussagen
- 50 eingeschlossene Studien zu LLM-Anwendungen in der Pflege
- 56 evaluative Elemente in 3 COMET-Domänen
- Messungen zu Datenextraktion und Decision Support waren begrenzt
Deutsche Zusammenfassung: KI-gestützt aus dem Original-Abstract erstellt; alle Zahlen wurden automatisch mit dem Original abgeglichen. Maßgeblich ist der Original-Abstract.
Kurz gefasst
Scoping Review in der Zeitschrift International Nursing Review, online erschienen am 11.09.2026. Verfasst von Zhiwei Lin und 5 weiteren Autor:innen. Beteiligte Einrichtung: School of Nursing, Fujian University of Traditional Chinese Medicine, Fuzhou, China. Hauptthemen laut MEDLINE: Large Language Models, Outcome Assessment, Health Care. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 69 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.
Abstract (Originalsprache)
Aim
To systematically identify and categorize the outcomes, evaluation metrics, and measurement tools employed in studies evaluating the application of large language models (LLMs) in the nursing field.
Background
LLMs are increasingly applied in nursing education, clinical practice, and research. However, outcomes, evaluation metrics, and measurement tools used to evaluate these applications remain heterogeneous and inconsistently reported, limiting cross‐study comparability and evidence synthesis.
Methods
This scoping review was conducted in accordance with the Joanna Briggs Institute methodology and PRISMA‐ScR guidelines. Outcomes, evaluation metrics, and measurement tools were extracted and categorized according to the Consensus‐based Method for the Selection of Health Outcome Measurement Tools (COMET) framework domains, then mapped to LLM task types based on the Trustworthiness and Reporting of Items for Prediction Model Development–Large Language Models (TRIPOD‐LLM) guidelines. Sources of Evidence Eight English and Chinese databases were searched to identify original empirical studies evaluating LLM applications in nursing, yielding 50 studies that met the inclusion criteria.
Results
A total of 56 distinct evaluative elements (encompassing clinical/educational outcomes, technical evaluation metrics, and standardized measurement tools) were extracted and categorized into three COMET‐derived domains: objective efficiency indicators, intermediate effect indicators, and subjective experience indicators. Most measures focused on text generation and summarization tasks, while evaluation metrics for data extraction and decision support tasks were relatively limited. This reflects issues of reporting inconsistency and insufficient standardization.
Discussion
Performance metrics for LLMs in nursing applications exhibit uneven distribution across task modalities and operational scenarios, underscoring the necessity for standardized evaluation guidelines.
Conclusion
This evidence‐based review supports greater consistency in reporting and organizing LLM applications within nursing research and practice.
Implications for Nursing
Practice The findings of this review can support nurses and educators in selecting appropriate evaluation indicators, promoting safe and evidence‐informed integration of LLMs.
Implications for Nursing
Policy When evaluating LLM applications in nursing, enhancing the clarity and consistency of outcome metric reporting facilitates the development of institutional‐level guidelines.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- School of Nursing, Fujian University of Traditional Chinese Medicine, Fuzhou, China.
- The Seventh Clinical College of Guangzhou University of Chinese Medicine, Shenzhen, China.
ORCID-Profile: Zhiwei Lin, Xinglin Zheng
Zitieren
APA
Lin, Z., Huang, X., Yan, Y., Zheng, X., Wang, W., & Su, G. (2026). Outcomes, Evaluation Metrics, and Measurement Tools for Large Language Model Applications Research in Nursing: A Scoping Review. International Nursing Review, 73(3). https://doi.org/10.1111/inr.70217
Vancouver
Lin Z, Huang X, Yan Y, Zheng X, Wang W, Su G. Outcomes, Evaluation Metrics, and Measurement Tools for Large Language Model Applications Research in Nursing: A Scoping Review. International Nursing Review. 2026;73(3). doi:10.1111/inr.70217
BibTeX
@article{Lin2026,
author = {Lin, Zhiwei and Huang, Xiaoying and Yan, Yanhong and Zheng, Xinglin and Wang, Wenyu and Su, Guang},
title = {Outcomes, Evaluation Metrics, and Measurement Tools for Large Language Model Applications Research in Nursing: A Scoping Review},
journal = {International Nursing Review},
year = {2026},
volume = {73},
number = {3},
doi = {10.1111/inr.70217}
}
EUVIMED