Machine Learning Facial Emotion Classifiers in Psychotherapy Research: A Proof-of-Concept Study
Psychopathology · 2024 · 57 (3) · 159-168
DOI: 10.1159/000534811 ↗Kurz gefasst
Fachartikel in der Zeitschrift Psychopathology, online erschienen am 27.11.2023. Verfasst von Martin Steppan und 8 weiteren Autor:innen. Beteiligte Einrichtung: Faculty of Psychology, University of Basel, Basel, Switzerland. Hauptthemen laut MEDLINE: Machine Learning, Psychotherapy, Emotions, Facial Expression. Die Arbeit wurde bisher 11-mal zitiert (Crossref). Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0).
Abstract (Originalsprache)
Background
New advances in the field of machine learning make it possible to track facial emotional expression with high resolution, including micro-expressions. These advances have promising applications for psychotherapy research, since manual coding (e.g., the Facial Action Coding System), is time-consuming.
Purpose
We tested whether this technology can reliably identify in-session emotional expression in a naturalistic treatment setting, and how these measures relate to the outcome of psychotherapy.
Method
We applied a machine learning emotion classifier to video material from 389 psychotherapy sessions of 23 patients with borderline personality pathology. We validated the findings with human ratings according to the Clients Emotional Arousal Scale (CEAS) and explored associations with treatment outcomes.
Results
Overall, machine learning ratings showed significant agreement with human ratings. Machine learning emotion classifiers, particularly the display of positive emotions (smiling and happiness), showed medium effect size on median-split treatment outcome (d = 0.3) as well as continuous improvement (r = 0.49, p < 0.05). Patients who dropped out form psychotherapy, showed significantly more neutral expressions, and generally less social smiling, particularly at the beginning of psychotherapeutic sessions.
Conclusions
Machine learning classifiers are a highly promising resource for research in psychotherapy. The results highlight differential associations of displayed positive and negative feelings with treatment outcomes. Machine learning emotion recognition may be used for the early identification of drop-out risks and clinically relevant interactions in psychotherapy.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Faculty of Psychology, University of Basel, Basel, Switzerland.
- Psychiatric University Hospital, Basel, Switzerland.
- Department of Psychology, University of Kentucky, Lexington, Kentucky, USA.
- University Hospital of Child and Adolescent Psychiatry and Psychotherapy, University of Bern, Bern, Switzerland.
- Section for Experimental Child and Adolescent Psychiatry, Department of Child and Adolescent Psychiatry, Centre for Psychosocial Medicine, University of Heidelberg, Heidelberg, Germany.
- Section for Translational Psychobiology in Child and Adolescent Psychiatry, Department of Child and Adolescent Psychiatry, Centre for Psychosocial Medicine, University of Heidelberg, Heidelberg, Germany.
Zitieren
APA
Steppan, M., Zimmermann, R., Fürer, L., Southward, M., Koenig, J., Kaess, M., Kleinbub, J. R., Roth, V., & Schmeck, K. (2024). Machine Learning Facial Emotion Classifiers in Psychotherapy Research: A Proof-of-Concept Study. Psychopathology, 57(3), 159-168. https://doi.org/10.1159/000534811
Vancouver
Steppan M, Zimmermann R, Fürer L, Southward M, Koenig J, Kaess M, et al. Machine Learning Facial Emotion Classifiers in Psychotherapy Research: A Proof-of-Concept Study. Psychopathology. 2024;57(3):159-168. doi:10.1159/000534811
BibTeX
@article{Steppan2024,
author = {Steppan, Martin and Zimmermann, Ronan and Fürer, Lukas and Southward, Matthew and Koenig, Julian and Kaess, Michael and Kleinbub, Johann Roland and Roth, Volker and Schmeck, Klaus},
title = {Machine Learning Facial Emotion Classifiers in Psychotherapy Research: A Proof-of-Concept Study},
journal = {Psychopathology},
year = {2024},
volume = {57},
number = {3},
pages = {159--168},
doi = {10.1159/000534811}
}
EUVIMED