EUVIMEDOpdag EUVIMED
Open access

An Interpretable Model With Probabilistic Integrated Scoring for Mental Health Treatment Prediction: Design Study

Anthony Kelly iD, Esben Kjems Jensen iD, Eoin Martino Grua iD, Kim Mathiasen iD, Pepijn Van de Ven iD

JMIR Medical Informatics · 2025

DOI: 10.2196/64617 ↗

At a glance

Journal article in JMIR Medical Informatics, 2025. Written by Anthony Kelly and 4 co-authors. Institution: Department of Electronic and Computer Engineering, University of Limerick, Limerick, Ireland. Major MEDLINE topics: Machine Learning, Mental Disorders, Models, Statistical, Decision Support Systems, Clinical. Cited 11 times so far (Crossref); 59 references. The full text is freely available in PubMed Central.

AbstractSubjectsBibliografiske oplysningerCiteRelated studies

Abstract (original language)

Background

Machine learning (ML) systems in health care have the potential to enhance decision-making but often fail to address critical issues such as prediction explainability, confidence, and robustness in a context-based and easily interpretable manner.

Objective

This study aimed to design and evaluate an ML model for a future decision support system for clinical psychopathological treatment assessments. The novel ML model is inherently interpretable and transparent. It aims to enhance clinical explainability and trust through a transparent, hierarchical model structure that progresses from questions to scores to classification predictions. The model confidence and robustness were addressed by applying Monte Carlo dropout, a probabilistic method that reveals model uncertainty and confidence.

Methods

A model for clinical psychopathological treatment assessments was developed, incorporating a novel ML model structure. The model aimed at enhancing the graphical interpretation of the model outputs and addressing issues of prediction explainability, confidence, and robustness. The proposed ML model was trained and validated using patient questionnaire answers and demographics from a web-based treatment service in Denmark (N=1088).

Results

The balanced accuracy score on the test set was 0.79. The precision was ≥0.71 for all 4 prediction classes (depression, panic, social phobia, and specific phobia). The area under the curve for the 4 classes was 0.93, 0.92, 0.91, and 0.98, respectively.

Conclusions

We have demonstrated a mental health treatment ML model that supported a graphical interpretation of prediction class probability distributions. Their spread and overlap can inform clinicians of competing treatment possibilities for patients and uncertainty in treatment predictions. With the ML model achieving 79% balanced accuracy, we expect that the model will be clinically useful in both screening new patients and informing clinical interviews.

Subjects

MeSH terms (MEDLINE)

Machine LearningMental DisordersModels, StatisticalDecision Support Systems, ClinicalHumansMaleFemaleDenmarkAdultSurveys and QuestionnairesMiddle AgedMental Health

Author keywords

AIMonte Carlo dropoutXAIartificial intelligenceexplainabilityexplainable AImachine learningmental health

Authors and institutions

  • Department of Electronic and Computer Engineering, University of Limerick, Limerick, Ireland.
  • Health Research Institute, University of Limerick, Limerick, Ireland.
  • Center for Digital Psykiatri, Forskningsenhed, Denmark.
  • Department of Psychology and Behavioural Sciences, Aarhus University, Aarhus, Denmark.
  • Department of Clinical Research, University of Southern Denmark, Odense, Denmark.

ORCID profiles: Anthony Kelly, Esben Kjems Jensen, Eoin Martino Grua, Kim Mathiasen, Pepijn Van de Ven

Cite

APA

Kelly, A., Jensen, E. K., Grua, E. M., Mathiasen, K., & Van de Ven, P. (2025). An Interpretable Model With Probabilistic Integrated Scoring for Mental Health Treatment Prediction: Design Study. JMIR Medical Informatics. https://doi.org/10.2196/64617

Vancouver

Kelly A, Jensen EK, Grua EM, Mathiasen K, Van de Ven P. An Interpretable Model With Probabilistic Integrated Scoring for Mental Health Treatment Prediction: Design Study. JMIR Medical Informatics. 2025. doi:10.2196/64617

BibTeX
@article{Kelly2025,
  author = {Kelly, Anthony and Jensen, Esben Kjems and Grua, Eoin Martino and Mathiasen, Kim and Van de Ven, Pepijn},
  title = {An Interpretable Model With Probabilistic Integrated Scoring for Mental Health Treatment Prediction: Design Study},
  journal = {JMIR Medical Informatics},
  year = {2025},
  doi = {10.2196/64617}
}
Start for free →