EUVIMEDEUVIMED entdecken
Crossref

Hierarchical disentangled representation learning for multimodal and multi-task pilot cognitive state monitoring

Jun Yao, Lixin Han, Xizhong Yang, Yiqun Dong

Biomedical Signal Processing and Control · 2027-02 · 130 · 111582

DOI: 10.1016/j.bspc.2026.111582 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Biomedical Signal Processing and Control, 2027-02. Verfasst von Jun Yao und 3 weiteren Autor:innen. Die Arbeit nennt 46 Literaturangaben. Gefördert von: National Natural Science Foundation of China.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

During complex flight operations, multiple cognitive states may coexist and jointly influence pilot performance and flight safety. However, most existing pilot cognitive-state monitoring methods either model individual cognitive dimensions independently or fuse heterogeneous sensing modalities without explicitly distinguishing shared from modality-specific information, limiting their ability to exploit both cross-modal complementarity and cross-task common structure. To address these challenges, a Hierarchical Modality–Task Disentanglement Network (HMTD-Net) is proposed for multimodal and multi-task pilot cognitive-state monitoring. HMTD-Net performs shared–specific representation decomposition sequentially at two levels. At the modality level, modality-shared and modality-specific representations are learned under distributional similarity, orthogonality, and reconstruction constraints and subsequently integrated through multi-head self-attention. At the task level, the fused multimodal representation is further decomposed into task-shared and task-specific components. The task-shared representations form a common memory, from which each task selectively retrieves relevant shared information through cross-attention while retaining its task-specific representation. Experiments on the CLV-Pilot and MODALSIM94 datasets under both subject-dependent and subject-independent protocols show that HMTD-Net consistently outperforms representative single-task and multi-task baselines. In particular, on MODALSIM94, HMTD-Net achieves 89.52% situational-awareness recognition accuracy under the subject-dependent protocol, outperforming the strongest competing multi-task method by 2.29%; under the subject-independent protocol, it achieves 75.85% workload Macro-F1, exceeding the strongest competitor by 3.87%. Ablation studies further demonstrate consistent contributions from both levels of shared–specific decomposition and their associated constraints. These results establish HMTD-Net as an effective framework for jointly recognizing multiple pilot cognitive dimensions from heterogeneous multimodal signals.

Förderung

  • National Natural Science Foundation of China

Zitieren

APA

Yao, J., Han, L., Yang, X., & Dong, Y. (2027). Hierarchical disentangled representation learning for multimodal and multi-task pilot cognitive state monitoring. Biomedical Signal Processing and Control, 130, 111582. https://doi.org/10.1016/j.bspc.2026.111582

Vancouver

Yao J, Han L, Yang X, Dong Y. Hierarchical disentangled representation learning for multimodal and multi-task pilot cognitive state monitoring. Biomedical Signal Processing and Control. 2027;130:111582. doi:10.1016/j.bspc.2026.111582

BibTeX
@article{Yao2027,
  author = {Yao, Jun and Han, Lixin and Yang, Xizhong and Dong, Yiqun},
  title = {Hierarchical disentangled representation learning for multimodal and multi-task pilot cognitive state monitoring},
  journal = {Biomedical Signal Processing and Control},
  year = {2027},
  volume = {130},
  pages = {111582},
  doi = {10.1016/j.bspc.2026.111582}
}
Kostenlos starten →