Hierarchical disentangled representation learning for multimodal and multi-task pilot cognitive state monitoring
Biomedical Signal Processing and Control · 2027-02 · 130 · 111582
DOI: 10.1016/j.bspc.2026.111582 ↗Kurz gefasst
Fachartikel in der Zeitschrift Biomedical Signal Processing and Control, 2027-02. Verfasst von Jun Yao und 3 weiteren Autor:innen. Die Arbeit nennt 46 Literaturangaben. Gefördert von: National Natural Science Foundation of China.
Abstract (Originalsprache)
During complex flight operations, multiple cognitive states may coexist and jointly influence pilot performance and flight safety. However, most existing pilot cognitive-state monitoring methods either model individual cognitive dimensions independently or fuse heterogeneous sensing modalities without explicitly distinguishing shared from modality-specific information, limiting their ability to exploit both cross-modal complementarity and cross-task common structure. To address these challenges, a Hierarchical Modality–Task Disentanglement Network (HMTD-Net) is proposed for multimodal and multi-task pilot cognitive-state monitoring. HMTD-Net performs shared–specific representation decomposition sequentially at two levels. At the modality level, modality-shared and modality-specific representations are learned under distributional similarity, orthogonality, and reconstruction constraints and subsequently integrated through multi-head self-attention. At the task level, the fused multimodal representation is further decomposed into task-shared and task-specific components. The task-shared representations form a common memory, from which each task selectively retrieves relevant shared information through cross-attention while retaining its task-specific representation. Experiments on the CLV-Pilot and MODALSIM94 datasets under both subject-dependent and subject-independent protocols show that HMTD-Net consistently outperforms representative single-task and multi-task baselines. In particular, on MODALSIM94, HMTD-Net achieves 89.52% situational-awareness recognition accuracy under the subject-dependent protocol, outperforming the strongest competing multi-task method by 2.29%; under the subject-independent protocol, it achieves 75.85% workload Macro-F1, exceeding the strongest competitor by 3.87%. Ablation studies further demonstrate consistent contributions from both levels of shared–specific decomposition and their associated constraints. These results establish HMTD-Net as an effective framework for jointly recognizing multiple pilot cognitive dimensions from heterogeneous multimodal signals.
Förderung
- National Natural Science Foundation of China
Zitieren
APA
Yao, J., Han, L., Yang, X., & Dong, Y. (2027). Hierarchical disentangled representation learning for multimodal and multi-task pilot cognitive state monitoring. Biomedical Signal Processing and Control, 130, 111582. https://doi.org/10.1016/j.bspc.2026.111582
Vancouver
Yao J, Han L, Yang X, Dong Y. Hierarchical disentangled representation learning for multimodal and multi-task pilot cognitive state monitoring. Biomedical Signal Processing and Control. 2027;130:111582. doi:10.1016/j.bspc.2026.111582
BibTeX
@article{Yao2027,
author = {Yao, Jun and Han, Lixin and Yang, Xizhong and Dong, Yiqun},
title = {Hierarchical disentangled representation learning for multimodal and multi-task pilot cognitive state monitoring},
journal = {Biomedical Signal Processing and Control},
year = {2027},
volume = {130},
pages = {111582},
doi = {10.1016/j.bspc.2026.111582}
}
EUVIMED