EUVIMEDEUVIMED entdecken
Open Access

Empowering Mental Health Monitoring Using a Macro-Micro Personalization Framework for Multimodal-Multitask Learning: Descriptive Study

Meishu Song iD, Zijiang Yang iD, Andreas Triantafyllopoulos iD, Zixing Zhang iD, Zhe Nan iD, Muxuan Tang iD, Hiroki Takeuchi iD, Toru Nakamura iD, Akifumi Kishi iD, Tetsuro Ishizawa iD, Kazuhiro Yoshiuchi iD, Björn Schuller iD und 1 weitere

JMIR Mental Health · 18.10.2024 · 11 · e59512

DOI: 10.2196/59512 ↗

Kurz gefasst

Fachartikel in der Zeitschrift JMIR Mental Health, online erschienen am 18.10.2024. Verfasst von Meishu Song und 12 weiteren Autor:innen. Beteiligte Einrichtung: Educational Physiology Laboratory, Graduate School of Education, The University of Tokyo, Tokyo, Japan. Hauptthemen laut MEDLINE: Smartphone. Die Arbeit wurde bisher 12-mal zitiert (Crossref) und nennt 55 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

The field of mental health technology presently has significant gaps that need addressing, particularly in the domain of daily monitoring and personalized assessments. Current noninvasive devices such as wristbands and smartphones are capable of collecting a wide range of data, which has not yet been fully used for mental health monitoring.

Objective

This study aims to introduce a novel dataset for personalized daily mental health monitoring and a new macro-micro framework. This framework is designed to use multimodal and multitask learning strategies for improved personalization and prediction of emotional states in individuals.

Methods

Data were collected from 298 individuals using wristbands and smartphones, capturing physiological signals, speech data, and self-annotated emotional states. The proposed framework combines macro-level emotion transformer embeddings with micro-level personalization layers specific to each user. It also introduces a Dynamic Restrained Uncertainty Weighting method to effectively integrate various data types for a balanced representation of emotional states. Several fusion techniques, personalization strategies, and multitask learning approaches were explored.

Results

The proposed framework was evaluated using the concordance correlation coefficient, resulting in a score of 0.503. This result demonstrates the framework’s efficacy in predicting emotional states.

Conclusions

The study concludes that the proposed multimodal and multitask learning framework, which leverages transformer-based techniques and dynamic task weighting strategies, is superior for the personalized monitoring of mental health. The study indicates the potential of transforming daily mental health monitoring into a more personalized app, opening up new avenues for technology-based mental health interventions.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

SmartphoneHumansMaleFemaleAdultMiddle AgedEmotionsMental HealthYoung AdultWearable Electronic DevicesEmpowermentAdolescent

Stichwörter der Autor:innen

daily mental healthemotional stateframeworkmacromental healthmicromobile phonesmonitoringmultimodalmultitaskpersonalizationphysiologicalpredictionsignalssmartphone

Autor:innen und Institutionen

  • Educational Physiology Laboratory, Graduate School of Education, The University of Tokyo, Tokyo, Japan.
  • Technical University of Munich, Munich, Germany.
  • Hunan University, Changsha, China.
  • Google, San Carlos, CA, United States.
  • Osaka University, Osaka, Japan.
  • Imperial College London, London, United Kingdom.

ORCID-Profile: Meishu Song, Zijiang Yang, Andreas Triantafyllopoulos, Zixing Zhang, Zhe Nan, Muxuan Tang, Hiroki Takeuchi, Toru Nakamura, Akifumi Kishi, Tetsuro Ishizawa, Kazuhiro Yoshiuchi, Björn Schuller, Yoshiharu Yamamoto

Zitieren

APA

Song, M., Yang, Z., Triantafyllopoulos, A., Zhang, Z., Nan, Z., Tang, M., Takeuchi, H., Nakamura, T., Kishi, A., Ishizawa, T., Yoshiuchi, K., Schuller, B., & Yamamoto, Y. (2024). Empowering Mental Health Monitoring Using a Macro-Micro Personalization Framework for Multimodal-Multitask Learning: Descriptive Study. JMIR Mental Health, 11, e59512. https://doi.org/10.2196/59512

Vancouver

Song M, Yang Z, Triantafyllopoulos A, Zhang Z, Nan Z, Tang M, et al. Empowering Mental Health Monitoring Using a Macro-Micro Personalization Framework for Multimodal-Multitask Learning: Descriptive Study. JMIR Mental Health. 2024;11:e59512. doi:10.2196/59512

BibTeX
@article{Song2024,
  author = {Song, Meishu and Yang, Zijiang and Triantafyllopoulos, Andreas and Zhang, Zixing and Nan, Zhe and Tang, Muxuan and Takeuchi, Hiroki and Nakamura, Toru and Kishi, Akifumi and Ishizawa, Tetsuro and Yoshiuchi, Kazuhiro and Schuller, Björn and Yamamoto, Yoshiharu},
  title = {Empowering Mental Health Monitoring Using a Macro-Micro Personalization Framework for Multimodal-Multitask Learning: Descriptive Study},
  journal = {JMIR Mental Health},
  year = {2024},
  volume = {11},
  pages = {e59512},
  doi = {10.2196/59512}
}
Kostenlos starten →