Evaluating and mitigating unfairness in multimodal remote mental health assessments
PLOS Digital Health · 24.07.2024 · 3 (7) · e0000413
DOI: 10.1371/journal.pdig.0000413 ↗Kurz gefasst
Fachartikel in der Zeitschrift PLOS Digital Health, online erschienen am 24.07.2024. Verfasst von Zifan Jiang und 7 weiteren Autor:innen. Beteiligte Einrichtung: Department of Biomedical Informatics, Emory University School of Medicine, Atlanta, Georgia, United States of America. Die Arbeit wurde bisher 11-mal zitiert (Crossref) und nennt 68 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Georgia Clinical and Translational Science Alliance.
Abstract (Originalsprache)
Research on automated mental health assessment tools has been growing in recent years, often aiming to address the subjectivity and bias that existed in the current clinical practice of the psychiatric evaluation process. Despite the substantial health and economic ramifications, the potential unfairness of those automated tools was understudied and required more attention. In this work, we systematically evaluated the fairness level in a multimodal remote mental health dataset and an assessment system, where we compared the fairness level in race, gender, education level, and age. Demographic parity ratio (DPR) and equalized odds ratio (EOR) of classifiers using different modalities were compared, along with the F1 scores in different demographic groups. Post-training classifier threshold optimization was employed to mitigate the unfairness. No statistically significant unfairness was found in the composition of the dataset. Varying degrees of unfairness were identified among modalities, with no single modality consistently demonstrating better fairness across all demographic variables. Post-training mitigation effectively improved both DPR and EOR metrics at the expense of a decrease in F1 scores. Addressing and mitigating unfairness in these automated tools are essential steps in fostering trust among clinicians, gaining deeper insights into their use cases, and facilitating their appropriate utilization.
Autor:innen und Institutionen
- Department of Biomedical Informatics, Emory University School of Medicine, Atlanta, Georgia, United States of America.
- Department of Biomedical Engineering, Emory University and Georgia Institute of Technology, Atlanta, Georgia, United States of America.
- Department of Psychiatry and Behavioral Sciences, Emory University School of Medicine, Atlanta, Georgia, United States of America.
- Department of IT, Analytics, and Operations, University of Notre Dame, Notre Dame, Indiana, United States of America.
ORCID-Profile: Zifan Jiang
Förderung
- Georgia Clinical and Translational Science Alliance
Zitieren
APA
Jiang, Z., Seyedi, S., Griner, E., Abbasi, A., Rad, A. B., Kwon, H., Cotes, R. O., & Clifford, G. D. (2024). Evaluating and mitigating unfairness in multimodal remote mental health assessments. PLOS Digital Health, 3(7), e0000413. https://doi.org/10.1371/journal.pdig.0000413
Vancouver
Jiang Z, Seyedi S, Griner E, Abbasi A, Rad AB, Kwon H, et al. Evaluating and mitigating unfairness in multimodal remote mental health assessments. PLOS Digital Health. 2024;3(7):e0000413. doi:10.1371/journal.pdig.0000413
BibTeX
@article{Jiang2024,
author = {Jiang, Zifan and Seyedi, Salman and Griner, Emily and Abbasi, Ahmed and Rad, Ali Bahrami and Kwon, Hyeokhyen and Cotes, Robert O and Clifford, Gari D},
title = {Evaluating and mitigating unfairness in multimodal remote mental health assessments},
journal = {PLOS Digital Health},
year = {2024},
volume = {3},
number = {7},
pages = {e0000413},
doi = {10.1371/journal.pdig.0000413}
}
EUVIMED