EUVIMEDEUVIMED entdecken
Scoping ReviewOpen Access

Outcomes, Evaluation Metrics, and Measurement Tools for Large Language Model Applications Research in Nursing: A Scoping Review

Outcomes, Evaluation Metrics und Measurement Tools für Forschungsanwendungen von Large Language Models in der Pflege: Scoping Review

Zhiwei Lin iD, Xiaoying Huang, Yanhong Yan, Xinglin Zheng iD, Wenyu Wang, Guang Su

International Nursing Review · 2026-09 · 73 (3)

DOI: 10.1111/inr.70217 ↗

Auf einen Blick

Die Scoping Review identifizierte 50 Studien zu LLM-Anwendungen in der Pflege. Die berichteten Outcomes und Messinstrumente waren heterogen und unzureichend standardisiert.

Fragestellung

Welche Outcomes, Evaluationsmetriken und Messinstrumente wurden in Studien zu LLM-Anwendungen in der Pflege verwendet?

Methode

Scoping Review nach Joanna Briggs Institute und PRISMA-ScR; 8 englische und chinesische Datenbanken; Einordnung nach COMET und TRIPOD-LLM.

Ergebnisse

50 Studien erfüllten die Einschlusskriterien. Es wurden 56 verschiedene evaluative Elemente extrahiert und drei COMET-Domänen zugeordnet: objektive Effizienzindikatoren, intermediäre Effekte und subjektive Erfahrung.

Einschränkungen

Heterogene und inkonsistent berichtete Outcomes, Metriken und Messinstrumente; geringe Standardisierung.

Schlussfolgerung der Autor:innen

Die Übersichtsarbeit unterstützt eine konsistentere Berichterstattung und Strukturierung von LLM-Anwendungen in Pflegeforschung und -praxis.

Kernaussagen

  • 50 eingeschlossene Studien zu LLM-Anwendungen in der Pflege
  • 56 evaluative Elemente in 3 COMET-Domänen
  • Messungen zu Datenextraktion und Decision Support waren begrenzt

Deutsche Zusammenfassung: KI-gestützt aus dem Original-Abstract erstellt; alle Zahlen wurden automatisch mit dem Original abgeglichen. Maßgeblich ist der Original-Abstract.

Kurz gefasst

Scoping Review in der Zeitschrift International Nursing Review, online erschienen am 11.09.2026. Verfasst von Zhiwei Lin und 5 weiteren Autor:innen. Beteiligte Einrichtung: School of Nursing, Fujian University of Traditional Chinese Medicine, Fuzhou, China. Hauptthemen laut MEDLINE: Large Language Models, Outcome Assessment, Health Care. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 69 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.

Auf einen BlickAbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Aim

To systematically identify and categorize the outcomes, evaluation metrics, and measurement tools employed in studies evaluating the application of large language models (LLMs) in the nursing field.

Background

LLMs are increasingly applied in nursing education, clinical practice, and research. However, outcomes, evaluation metrics, and measurement tools used to evaluate these applications remain heterogeneous and inconsistently reported, limiting cross‐study comparability and evidence synthesis.

Methods

This scoping review was conducted in accordance with the Joanna Briggs Institute methodology and PRISMA‐ScR guidelines. Outcomes, evaluation metrics, and measurement tools were extracted and categorized according to the Consensus‐based Method for the Selection of Health Outcome Measurement Tools (COMET) framework domains, then mapped to LLM task types based on the Trustworthiness and Reporting of Items for Prediction Model Development–Large Language Models (TRIPOD‐LLM) guidelines. Sources of Evidence Eight English and Chinese databases were searched to identify original empirical studies evaluating LLM applications in nursing, yielding 50 studies that met the inclusion criteria.

Results

A total of 56 distinct evaluative elements (encompassing clinical/educational outcomes, technical evaluation metrics, and standardized measurement tools) were extracted and categorized into three COMET‐derived domains: objective efficiency indicators, intermediate effect indicators, and subjective experience indicators. Most measures focused on text generation and summarization tasks, while evaluation metrics for data extraction and decision support tasks were relatively limited. This reflects issues of reporting inconsistency and insufficient standardization.

Discussion

Performance metrics for LLMs in nursing applications exhibit uneven distribution across task modalities and operational scenarios, underscoring the necessity for standardized evaluation guidelines.

Conclusion

This evidence‐based review supports greater consistency in reporting and organizing LLM applications within nursing research and practice.

Implications for Nursing

Practice The findings of this review can support nurses and educators in selecting appropriate evaluation indicators, promoting safe and evidence‐informed integration of LLMs.

Implications for Nursing

Policy When evaluating LLM applications in nursing, enhancing the clarity and consistency of outcome metric reporting facilitates the development of institutional‐level guidelines.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Large Language ModelsOutcome Assessment, Health CareHumans

Stichwörter der Autor:innen

COMET HandbookTRIPOD‐LLM Guidelinesevaluation metricslarge language modelsmeasurement toolsnursingoutcome measuresscoping review

Autor:innen und Institutionen

  • School of Nursing, Fujian University of Traditional Chinese Medicine, Fuzhou, China.
  • The Seventh Clinical College of Guangzhou University of Chinese Medicine, Shenzhen, China.

ORCID-Profile: Zhiwei Lin, Xinglin Zheng

Zitieren

APA

Lin, Z., Huang, X., Yan, Y., Zheng, X., Wang, W., & Su, G. (2026). Outcomes, Evaluation Metrics, and Measurement Tools for Large Language Model Applications Research in Nursing: A Scoping Review. International Nursing Review, 73(3). https://doi.org/10.1111/inr.70217

Vancouver

Lin Z, Huang X, Yan Y, Zheng X, Wang W, Su G. Outcomes, Evaluation Metrics, and Measurement Tools for Large Language Model Applications Research in Nursing: A Scoping Review. International Nursing Review. 2026;73(3). doi:10.1111/inr.70217

BibTeX
@article{Lin2026,
  author = {Lin, Zhiwei and Huang, Xiaoying and Yan, Yanhong and Zheng, Xinglin and Wang, Wenyu and Su, Guang},
  title = {Outcomes, Evaluation Metrics, and Measurement Tools for Large Language Model Applications Research in Nursing: A Scoping Review},
  journal = {International Nursing Review},
  year = {2026},
  volume = {73},
  number = {3},
  doi = {10.1111/inr.70217}
}
Kostenlos starten →