EUVIMEDEUVIMED entdecken
Open Access

Refining a Machine Learning Model for Predicting Infant Sepsis: A Multidisciplinary Team Supported by Human-Centered Design Methods

Verfeinerung eines Machine-Learning-Modells zur Vorhersage von Säuglingssepsis: Ein multidisziplinäres Team unterstützt durch Human-Centered-Design-Methoden

Dean Karavite, Lusha Cao, Mary C Harris, Alex Fidel, Lyle Ungar, Gerald Shaeffer, Rui Xiao, Patrick Brady, Heather C Kaplan, Robert W Grundmeier

Applied Clinical Informatics · 01.08.2025 · 16 (04) · 1332-1340

DOI: 10.1055/a-2618-4470 ↗

Auf einen Blick

HCD-Methoden deckten bei einem ML-Modell zur Sepsis-Erkennung bei Neugeborenen Anomalien auf. Daraus ergaben sich 41 Änderungen am Modell.

Fragestellung

Wie können Human-Centered-Design-Methoden die Entwicklung und Prüfung eines Machine-Learning-Modells zur Sepsis-Erkennung im NICU-Bereich unterstützen?

Methode

Fallbericht mit qualitativer Interviewanalyse von 31 NICU-Kliniker:innen; zusätzlich Low-Fidelity-Mockups, Chart- und Code-Inspektion durch ein multidisziplinäres Team.

Ergebnisse

Die HCD-gestützte Prüfung identifizierte Anomalien zu Feature-Ein- und -Ausschluss, Feature-Wichtigkeit und Modellstabilität über die Zeit. Auch Dateneingabefehler im EHR wurden erkannt. Es wurden 41 Änderungen am Modell vorgenommen.

Schlussfolgerung der Autor:innen

Die Autor:innen schließen, dass HCD nicht nur für Darstellung und Erklärbarkeit, sondern auch für Entwicklung und Bewertung von ML-Modellen genutzt werden kann.

Kernaussagen

  • Anomalien wurden in drei Kategorien gefunden
  • 31 NICU-Kliniker:innen wurden interviewt
  • 41 Änderungen am Modell folgten aus der Prüfung

Deutsche Zusammenfassung: KI-gestützt aus dem Original-Abstract erstellt; alle Zahlen wurden automatisch mit dem Original abgeglichen. Maßgeblich ist der Original-Abstract.

Kurz gefasst

Fachartikel in der Zeitschrift Applied Clinical Informatics, erschienen am 01.08.2025. Verfasst von Dean Karavite und 9 weiteren Autor:innen. Beteiligte Einrichtung: Department of Biomedical and Health Informatics, Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, United States. Hauptthemen laut MEDLINE: Machine Learning, Sepsis, Patient Care Team, Neonatal Sepsis. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 37 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: NLM NIH HHS, Foundation for the National Institutes of Health.

Auf einen BlickAbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Human-centered design (HCD) methods in machine learning generally focus on workflow, user interfaces, and data visualizations, but there is the potential to apply these methods to inform the model development and testing process. This study aimed to demonstrate the potential of HCD methods to support the design and testing of machine learning models developed for clinical decision-making. In preparing for formative user testing of clinician facing representations of a machine learning model for detecting sepsis in neonatal intensive care unit (NICU) patients, we discovered that interactive low fidelity mockups using real patient data revealed potential model anomalies. To further investigate these potential anomalies, we utilized the qualitative analysis of interviews with 31 NICU clinicians concerning their experience with neonatal sepsis. The review process was conducted by a multidisciplinary team with members having expertise in neonatology, informatics, data science, and human computer interaction (HCI). Anomalies identified via the mockups and interview analysis were further analyzed by inspections of patient charts and model features and code. The HCD-facilitated review revealed anomalies in three categories: (1) feature inclusion and exclusion, (2) feature importance, and (3) model stability over time. Data entry errors in the electronic health record and their impact on model output were also noted. The review resulted in 41 changes to the model. The discovery of over 41 opportunities to improve our prediction model was a serendipitous by-product of the HCD process. Our results suggest that HCD can be applied not only to model display design and measures of explainability, but to the development and evaluation of the model itself. This case report also demonstrates the need for a multidisciplinary team of clinicians, data scientists, and HCI experts in identifying and addressing issues involving machine learning model performance.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Machine LearningSepsisPatient Care TeamNeonatal SepsisHumansInfant, NewbornIntensive Care Units, NeonatalInfant

Autor:innen und Institutionen

  • Department of Biomedical and Health Informatics, Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, United States.
  • Department of Neonatology, Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, United States.
  • Department of Pediatrics, University of Pennsylvania Perelman School of Medicine, Philadelphia, Pennsylvania, United States.
  • Department of Computer and Information Science, University of Pennsylvania, Philadelphia, Pennsylvania, United States.
  • Department of Biostatistics, Epidemiology and Informatics, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, United States.
  • Cincinnati Children's Hospital Medical Center, Department of Pediatrics, Perinatal Institute and James M. Anderson Center for Health Systems Excellence, Cincinnati, Ohio, United States.

Förderung

  • NLM NIH HHS
  • Foundation for the National Institutes of Health

Zitieren

APA

Karavite, D., Cao, L., Harris, M. C., Fidel, A., Ungar, L., Shaeffer, G., Xiao, R., Brady, P., Kaplan, H. C., & Grundmeier, R. W. (2025). Refining a Machine Learning Model for Predicting Infant Sepsis: A Multidisciplinary Team Supported by Human-Centered Design Methods. Applied Clinical Informatics, 16(04), 1332-1340. https://doi.org/10.1055/a-2618-4470

Vancouver

Karavite D, Cao L, Harris MC, Fidel A, Ungar L, Shaeffer G, et al. Refining a Machine Learning Model for Predicting Infant Sepsis: A Multidisciplinary Team Supported by Human-Centered Design Methods. Applied Clinical Informatics. 2025;16(04):1332-1340. doi:10.1055/a-2618-4470

BibTeX
@article{Karavite2025,
  author = {Karavite, Dean and Cao, Lusha and Harris, Mary C and Fidel, Alex and Ungar, Lyle and Shaeffer, Gerald and Xiao, Rui and Brady, Patrick and Kaplan, Heather C and Grundmeier, Robert W},
  title = {Refining a Machine Learning Model for Predicting Infant Sepsis: A Multidisciplinary Team Supported by Human-Centered Design Methods},
  journal = {Applied Clinical Informatics},
  year = {2025},
  volume = {16},
  number = {04},
  pages = {1332--1340},
  doi = {10.1055/a-2618-4470}
}
Kostenlos starten →