Detecting Sociodemographic Biases in the Content and Quality of Large Language Model–Generated Nursing Care: Cross-Sectional Simulation Study
Erkennen soziodemografischer Verzerrungen in Inhalt und Qualität von Large-Language-Model-generierten Pflegeplänen: Querschnittliche Simulationsstudie
Journal of Medical Internet Research · 2025-12-05 · 27 · e78132-e78132
DOI: 10.2196/78132 ↗Auf einen Blick
GPT-4 erzeugte 9600 Pflegepläne für 96 Patientenprofile. Die Inhalte und die klinische Qualität unterschieden sich systematisch nach soziodemografischen Merkmalen.
Fragestellung
Unterscheiden sich von LLMs erzeugte Pflegepläne systematisch nach soziodemografischen Merkmalen der Patient:innen?
Methode
Gemischte Simulationsstudie mit standardisierten klinischen Vignetten; GPT-4 generierte 9600 Pflegepläne für 96 Patientenprofile. Quantitative Inhaltsanalyse aller Pläne und Expertenbewertung von 500 Plänen.
Ergebnisse
8 konsistente Themen. Bei niedrigem Einkommen seltener Environmental Adjustment (aRR 0,90), bei niedriger Bildung häufiger Family Support (aRR 1,10). Ältere Profile häufiger Pain Management (aRR 1,33) und Family Support (aRR 1,62), seltener Nurse Training (aRR 0,78).
Schlussfolgerung der Autor:innen
Die Autor:innen berichten, dass LLMs soziodemografische Verzerrungen in Pflegeplänen systematisch reproduzieren und diese sowohl Inhalt als auch Expertenurteile zur klinischen Qualität beeinflussen.
Kernaussagen
- Kommunikation und Education fast universell (99,98 %), Nurse Training am seltensten (39,3 %)
- Soziodemografische Merkmale beeinflussten die thematische Ausrichtung der Pflegepläne
- Expertenbewertung: hohe Gesamtqualität, aber Unterschiede bei Vollständigkeit, Anwendbarkeit und Sicherheit
Deutsche Zusammenfassung: KI-gestützt aus dem Original-Abstract erstellt; alle Zahlen wurden automatisch mit dem Original abgeglichen. Maßgeblich ist der Original-Abstract.
At a glance
Querschnittstudie in Journal of Medical Internet Research, published online on 2025-12-05. Written by Nan Bai and 12 co-authors. Institution: Center for Wise Information Technology of Mental Health Nursing Research, School of Nursing, Wuhan University, No. 115, Donghu Road, Wuchang District…. Major MEDLINE topics: Language, Nursing Care. Cited 13 times so far (Crossref); 54 references. The full text is freely available in PubMed Central.
Topics: Künstliche Intelligenz in der Pflege, Simulation in der Pflegeausbildung
Abstract (original language)
Background
Large language models (LLMs) are increasingly applied in health care. However, concerns remain that their nursing care recommendations may reflect patients’ sociodemographic attributes rather than clinical needs. While this risk is acknowledged, there is a lack of empirical evidence evaluating sociodemographic bias in LLM-generated nursing care plans.
Objective
To investigate potential biases in nursing care plans generated by LLMs, we focused on whether outputs differ systematically based on patients’ sociodemographic characteristics and assessed the implications for equitable nursing care.
Methods
We used a mixed methods simulation study. A standardized clinical vignette experiment was used to prompt GPT-4 to generate 9600 nursing care plans for 96 patient profiles with varying sociodemographic characteristics (eg, sex, age, income, education, and residence). We first conducted a quantitative analysis of all plans, assessing variations in thematic content. Subsequently, a panel of senior nursing experts evaluated the clinical quality (eg, safety, applicability, and completeness) of a stratified subsample of 500 plans.
Results
We analyzed 9600 LLM-generated nursing care plans and identified 8 consistent themes. Communication and Education (99.98%) and Emotional Support (99.97%) were nearly universal, while Nurse Training and Event Analysis were least frequent (39.3%). Multivariable analyses revealed systematic sociodemographic disparities. Care plans generated for low-income patient profiles were less likely to include the theme Environmental Adjustment (adjusted relative risk [aRR] 0.90). Profiles with lower education were associated with an increased likelihood of including Family Support (aRR 1.10). Similarly, plans generated for older patient profiles were more likely to contain recommendations for Pain Management (aRR 1.33) and Family Support (aRR 1.62) but were less likely to mention Nurse Training (aRR 0.78). Sex and regional differences were also significant. Expert review of 500 plans showed high overall quality (mean 4.47), with strong interrater reliability (κ=0.76‐0.81). However, urban profiles had higher completeness ( β =.22) and applicability ( β =.14) but lower safety scores (β=–0.09). These findings demonstrate that LLM-generated care plans exhibit systematic sociodemographic bias, raising important implications for fairness and safe deployment in nursing practice.
Conclusions
This study identified that LLMs systematically reproduce sociodemographic biases in the generation of nursing care plans. These biases appear in two forms: they shape the thematic content and influence expert-rated clinical quality. These findings reveal a substantial risk that such models may reinforce existing health inequities. To our knowledge, this is the first empirical evidence documenting these nuanced biases in nursing. The study also contributes a replicable framework for evaluating LLM-generated care plans. Finally, it underscores the critical need for robust human oversight to ensure that artificial intelligence serves as a tool for advancing equity rather than perpetuating disparities.
Subjects
MeSH terms (MEDLINE)
Author keywords
Authors and institutions
- Center for Wise Information Technology of Mental Health Nursing Research, School of Nursing, Wuhan University, No. 115, Donghu Road, Wuchang District, Wuhan, Hubei, China, +86 15902731922.
- Sorbonne Université, Institut du Cerveau - Paris Brain Institute - ICM, CNRS, Inria, Inserm, AP-HP, Hôpital de la Pitié Salpêtrière, Paris, France.
- College of Computer Science, Beijing University of Technology, Beijing, China.
- Department of Psychiatry, Renmin Hospital of Wuhan University, Wuhan, China.
- Research Center for Lifespan Health, Wuhan University, Wuhan, China.
Cite
APA
Bai, N., Yu, Y., Luo, C., Zhou, S. C., Wang, Q., Zou, H., Liu, Q., Fu, G., Zhai, W., Zhao, Q., Li, J., Wei, X., & Yang, B. X. (2025). Detecting Sociodemographic Biases in the Content and Quality of Large Language Model–Generated Nursing Care: Cross-Sectional Simulation Study. Journal of Medical Internet Research, 27, e78132-e78132. https://doi.org/10.2196/78132
Vancouver
Bai N, Yu Y, Luo C, Zhou SC, Wang Q, Zou H, et al. Detecting Sociodemographic Biases in the Content and Quality of Large Language Model–Generated Nursing Care: Cross-Sectional Simulation Study. Journal of Medical Internet Research. 2025;27:e78132-e78132. doi:10.2196/78132
BibTeX
@article{Bai2025,
author = {Bai, Nan and Yu, Yijing and Luo, Chunyan and Zhou, Si Chen and Wang, Qing and Zou, Huijing and Liu, Qian and Fu, Guanghui and Zhai, Wei and Zhao, Qing and Li, Jianqiang and Wei, Xinni and Yang, Bing Xiang},
title = {Detecting Sociodemographic Biases in the Content and Quality of Large Language Model–Generated Nursing Care: Cross-Sectional Simulation Study},
journal = {Journal of Medical Internet Research},
year = {2025},
volume = {27},
pages = {e78132--e78132},
doi = {10.2196/78132}
}
EUVIMED