EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC 4.0

Machine Learning Facial Emotion Classifiers in Psychotherapy Research: A Proof-of-Concept Study

Martin Steppan, Ronan Zimmermann, Lukas Fürer, Matthew Southward, Julian Koenig, Michael Kaess, Johann Roland Kleinbub, Volker Roth, Klaus Schmeck

Psychopathology · 2024 · 57 (3) · 159-168

DOI: 10.1159/000534811 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Psychopathology, online erschienen am 27.11.2023. Verfasst von Martin Steppan und 8 weiteren Autor:innen. Beteiligte Einrichtung: Faculty of Psychology, University of Basel, Basel, Switzerland. Hauptthemen laut MEDLINE: Machine Learning, Psychotherapy, Emotions, Facial Expression. Die Arbeit wurde bisher 11-mal zitiert (Crossref). Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

New advances in the field of machine learning make it possible to track facial emotional expression with high resolution, including micro-expressions. These advances have promising applications for psychotherapy research, since manual coding (e.g., the Facial Action Coding System), is time-consuming.

Purpose

We tested whether this technology can reliably identify in-session emotional expression in a naturalistic treatment setting, and how these measures relate to the outcome of psychotherapy.

Method

We applied a machine learning emotion classifier to video material from 389 psychotherapy sessions of 23 patients with borderline personality pathology. We validated the findings with human ratings according to the Clients Emotional Arousal Scale (CEAS) and explored associations with treatment outcomes.

Results

Overall, machine learning ratings showed significant agreement with human ratings. Machine learning emotion classifiers, particularly the display of positive emotions (smiling and happiness), showed medium effect size on median-split treatment outcome (d = 0.3) as well as continuous improvement (r = 0.49, p < 0.05). Patients who dropped out form psychotherapy, showed significantly more neutral expressions, and generally less social smiling, particularly at the beginning of psychotherapeutic sessions.

Conclusions

Machine learning classifiers are a highly promising resource for research in psychotherapy. The results highlight differential associations of displayed positive and negative feelings with treatment outcomes. Machine learning emotion recognition may be used for the early identification of drop-out risks and clinically relevant interactions in psychotherapy.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Machine LearningPsychotherapyEmotionsFacial ExpressionBorderline Personality DisorderHumansAdultFemaleMaleProof of Concept StudyMiddle AgedYoung AdultTreatment Outcome

Stichwörter der Autor:innen

AdolescentsBorderline personality disorderEmotionsFacial expressions classifiersPsychotherapy

Autor:innen und Institutionen

  • Faculty of Psychology, University of Basel, Basel, Switzerland.
  • Psychiatric University Hospital, Basel, Switzerland.
  • Department of Psychology, University of Kentucky, Lexington, Kentucky, USA.
  • University Hospital of Child and Adolescent Psychiatry and Psychotherapy, University of Bern, Bern, Switzerland.
  • Section for Experimental Child and Adolescent Psychiatry, Department of Child and Adolescent Psychiatry, Centre for Psychosocial Medicine, University of Heidelberg, Heidelberg, Germany.
  • Section for Translational Psychobiology in Child and Adolescent Psychiatry, Department of Child and Adolescent Psychiatry, Centre for Psychosocial Medicine, University of Heidelberg, Heidelberg, Germany.

Zitieren

APA

Steppan, M., Zimmermann, R., Fürer, L., Southward, M., Koenig, J., Kaess, M., Kleinbub, J. R., Roth, V., & Schmeck, K. (2024). Machine Learning Facial Emotion Classifiers in Psychotherapy Research: A Proof-of-Concept Study. Psychopathology, 57(3), 159-168. https://doi.org/10.1159/000534811

Vancouver

Steppan M, Zimmermann R, Fürer L, Southward M, Koenig J, Kaess M, et al. Machine Learning Facial Emotion Classifiers in Psychotherapy Research: A Proof-of-Concept Study. Psychopathology. 2024;57(3):159-168. doi:10.1159/000534811

BibTeX
@article{Steppan2024,
  author = {Steppan, Martin and Zimmermann, Ronan and Fürer, Lukas and Southward, Matthew and Koenig, Julian and Kaess, Michael and Kleinbub, Johann Roland and Roth, Volker and Schmeck, Klaus},
  title = {Machine Learning Facial Emotion Classifiers in Psychotherapy Research: A Proof-of-Concept Study},
  journal = {Psychopathology},
  year = {2024},
  volume = {57},
  number = {3},
  pages = {159--168},
  doi = {10.1159/000534811}
}
Kostenlos starten →