EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Generating Real‐World Evidence From AI Deployed in Health Systems

Lily Poursoltan iD, Aaron Boussina, Jie Cao, Eileen Kim iD, Poorva Satish Bedmutha iD, Hetvi Majmudar, Mattheus Ramsis iD, Alexander Khalessi, Karandeep Singh iD

Learning Health Systems · 2027-01 · 11 (1)

DOI: 10.1002/lrh2.70131 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Learning Health Systems, online erschienen am 04.11.2026. Verfasst von Lily Poursoltan und 8 weiteren Autor:innen. Beteiligte Einrichtung: Rady School of Management University of California San Diego California USA. Die Arbeit nennt 24 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Health systems increasingly deploy predictive and generative artificial intelligence ( AI ) to improve care delivery and outcomes. Predictive AI models estimate the risk of future events to inform targeted interventions, while generative AI systems extract, summarize, and communicate information and may automate selected tasks. Routine deployment generates clinical, operational, and interaction data that can support evaluation of the effectiveness and safety of AI ‐enabled workflows.

Methods

We review approaches to generating real‐world evidence ( RWE ) from routine deployment of predictive and generative AI across four implementation configurations: silent deployment, human‐in‐the‐loop ( HITL ), human‐on‐the‐loop ( HOTL ), and automated. For each configuration, we describe the evaluation questions, routinely generated data, and feasible study designs.

Results

Each implementation configuration supports different forms of evidence about model performance, workflow effectiveness, human oversight, and safety. For both predictive and generative AI , silent deployment supports assessment of local performance and subgroup differences. Once predictive AI drives action, evaluating the effectiveness of an implemented workflow ( HITL , HOTL , or automated) generally requires causal inference approaches. For generative AI , HITL implementations can generate data on edits, overrides, time on task, usability, cognitive burden, and safety‐relevant failure modes. HOTL and automated generative AI implementations can generate data on escalation performance, human follow‐up work, downstream outcomes, and safety signals.

Conclusion

AI implementation should be paired with methods for generating RWE on the effectiveness and safety of AI ‐enabled workflows. In HOTL and automated configurations, escalation performance and residual human workload become particularly important components of evaluation.

Schlagwörter

Stichwörter der Autor:innen

artificial intelligencegenerative AIimplementationlearning health systempredictive AIreal world evidence

Autor:innen und Institutionen

  • Rady School of Management University of California San Diego California USA.
  • Joan and Irwin Jacobs Center for Health Innovation UC San Diego Health San Diego California USA.
  • Division of Biomedical Informatics, Department of Medicine University of California San Diego California USA.
  • Division of General Internal Medicine, Department of Medicine University of California San Diego California USA.
  • Department of Electrical and Computer Engineering University of California San Diego California USA.
  • Department of Biology University of California San Diego California USA.

ORCID-Profile: Lily Poursoltan, Eileen Kim, Poorva Satish Bedmutha, Mattheus Ramsis, Karandeep Singh

Zitieren

APA

Poursoltan, L., Boussina, A., Cao, J., Kim, E., Bedmutha, P. S., Majmudar, H., Ramsis, M., Khalessi, A., & Singh, K. (2027). Generating Real‐World Evidence From AI Deployed in Health Systems. Learning Health Systems, 11(1). https://doi.org/10.1002/lrh2.70131

Vancouver

Poursoltan L, Boussina A, Cao J, Kim E, Bedmutha PS, Majmudar H, et al. Generating Real‐World Evidence From AI Deployed in Health Systems. Learning Health Systems. 2027;11(1). doi:10.1002/lrh2.70131

BibTeX
@article{Poursoltan2027,
  author = {Poursoltan, Lily and Boussina, Aaron and Cao, Jie and Kim, Eileen and Bedmutha, Poorva Satish and Majmudar, Hetvi and Ramsis, Mattheus and Khalessi, Alexander and Singh, Karandeep},
  title = {Generating Real‐World Evidence From AI Deployed in Health Systems},
  journal = {Learning Health Systems},
  year = {2027},
  volume = {11},
  number = {1},
  doi = {10.1002/lrh2.70131}
}
Kostenlos starten →