EUVIMEDEUVIMED entdecken
Open Access

Integrating Explainable Machine Learning in Clinical Decision Support Systems: Study Involving a Modified Design Thinking Approach

Michael Shulha iD, Jordan Hovdebo iD, Vinita D'Souza iD, Francis Thibault iD, Rola Harmouche iD

JMIR Formative Research · 16.04.2024 · 8 · e50475

DOI: 10.2196/50475 ↗

Kurz gefasst

Fachartikel in der Zeitschrift JMIR Formative Research, online erschienen am 16.04.2024. Verfasst von Michael Shulha und 4 weiteren Autor:innen. Beteiligte Einrichtung: Lady Davis Institute for Medical Research, Jewish General Hospital, Centre intégré universitaire de santé et de services sociaux (CIUSSS) du Centre-O…. Die Arbeit wurde bisher 24-mal zitiert (Crossref) und nennt 32 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Though there has been considerable effort to implement machine learning (ML) methods for health care, clinical implementation has lagged. Incorporating explainable machine learning (XML) methods through the development of a decision support tool using a design thinking approach is expected to lead to greater uptake of such tools.

Objective

This work aimed to explore how constant engagement of clinician end users can address the lack of adoption of ML tools in clinical contexts due to their lack of transparency and address challenges related to presenting explainability in a decision support interface.

Methods

We used a design thinking approach augmented with additional theoretical frameworks to provide more robust approaches to different phases of design. In particular, in the problem definition phase, we incorporated the nonadoption, abandonment, scale-up, spread, and sustainability of technology in health care (NASSS) framework to assess these aspects in a health care network. This process helped focus on the development of a prognostic tool that predicted the likelihood of admission to an intensive care ward based on disease severity in chest x-ray images. In the ideate, prototype, and test phases, we incorporated a metric framework to assess physician trust in artificial intelligence (AI) tools. This allowed us to compare physicians’ assessments of the domain representation, action ability, and consistency of the tool.

Results

Physicians found the design of the prototype elegant, and domain appropriate representation of data was displayed in the tool. They appreciated the simplified explainability overlay, which only displayed the most predictive patches that cumulatively explained 90% of the final admission risk score. Finally, in terms of consistency, physicians unanimously appreciated the capacity to compare multiple x-ray images in the same view. They also appreciated the ability to toggle the explainability overlay so that both options made it easier for them to assess how consistently the tool was identifying elements of the x-ray image they felt would contribute to overall disease severity.

Conclusions

The adopted approach is situated in an evolving space concerned with incorporating XML or AI technologies into health care software. We addressed the alignment of AI as it relates to clinician trust, describing an approach to wire framing and prototyping, which incorporates the use of a theoretical framework for trust in the design process itself. Moreover, we proposed that alignment of AI is dependent upon integration of end users throughout the larger design process. Our work shows the importance and value of engaging end users prior to tool development. We believe that the described approach is a unique and valuable contribution that outlines a direction for ML experts, user experience designers, and clinician end users on how to collaborate in the creation of trustworthy and usable XML-based clinical decision support tools.

Schlagwörter

Stichwörter der Autor:innen

COVID-19NASSS frameworkXMLchest x-rayclinical decision supportclinician engagementclinician trust in machine learningclinician-facing interfacedesign thinking approachexplainable machine learningseverity prediction

Autor:innen und Institutionen

  • Lady Davis Institute for Medical Research, Jewish General Hospital, Centre intégré universitaire de santé et de services sociaux (CIUSSS) du Centre-Ouest-de-l'Île-de-Montréal, Montreal, QC, Canada.
  • Department of Family Medicine, McGill University, Montreal, QC, Canada.
  • National Research Council of Canada, Winnipeg, MB, Canada.
  • National Research Council of Canada, Boucherville, QC, Canada.

ORCID-Profile: Michael Shulha, Jordan Hovdebo, Vinita D'Souza, Francis Thibault, Rola Harmouche

Zitieren

APA

Shulha, M., Hovdebo, J., D'Souza, V., Thibault, F., & Harmouche, R. (2024). Integrating Explainable Machine Learning in Clinical Decision Support Systems: Study Involving a Modified Design Thinking Approach. JMIR Formative Research, 8, e50475. https://doi.org/10.2196/50475

Vancouver

Shulha M, Hovdebo J, D'Souza V, Thibault F, Harmouche R. Integrating Explainable Machine Learning in Clinical Decision Support Systems: Study Involving a Modified Design Thinking Approach. JMIR Formative Research. 2024;8:e50475. doi:10.2196/50475

BibTeX
@article{Shulha2024,
  author = {Shulha, Michael and Hovdebo, Jordan and D'Souza, Vinita and Thibault, Francis and Harmouche, Rola},
  title = {Integrating Explainable Machine Learning in Clinical Decision Support Systems: Study Involving a Modified Design Thinking Approach},
  journal = {JMIR Formative Research},
  year = {2024},
  volume = {8},
  pages = {e50475},
  doi = {10.2196/50475}
}
Kostenlos starten →