Research on Safe Multimodal Detection Method of Pilot Visual Observation Behavior Based on Cognitive State Decoding
Multimodal Technologies and Interaction · 01.10.2025 · 9 (10) · 103
DOI: 10.3390/mti9100103 ↗Kurz gefasst
Fachartikel in der Zeitschrift Multimodal Technologies and Interaction, online erschienen am 01.10.2025. Verfasst von Heming Zhang und 2 weiteren Autor:innen. Die Arbeit wurde bisher 5-mal zitiert (Crossref) und nennt 42 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Natural Science Foundation of China, Key Research; Development Projects of Shaanxi Province, Key Laboratory of Metrological Optics and Application, State Administration for Market Regulation.
Abstract (Originalsprache)
Pilot visual behavior safety assessment is a cross-disciplinary technology that analyzes pilots’ gaze behavior and neurocognitive responses. This paper proposes a multimodal analysis method for pilot visual behavior safety, specifically for cognitive state decoding. This method aims to achieve a quantitative and efficient assessment of pilots’ observational behavior. Addressing the subjective limitations of traditional methods, this paper proposes an observational behavior detection model that integrates facial images to achieve dynamic and quantitative analysis of observational behavior. It addresses the “Midas contact” problem of observational behavior by constructing a cognitive analysis method using multimodal signals. We propose a bidirectional long short-term memory (LSTM) network that matches physiological signal rhythmic features to address the problem of isolated features in multidimensional signals. This method captures the dynamic correlations between multiple physiological behaviors, such as prefrontal theta and chest-abdominal coordination, to decode the cognitive state of pilots’ observational behavior. Finally, the paper uses a decision-level fusion method based on an improved Dempster–Shafer (DS) evidence theory to provide a quantifiable detection strategy for aviation safety standards. This dual-dimensional quantitative assessment system of “visual behavior–neurophysiological cognition” reveals the dynamic correlations between visual behavior and cognitive state among pilots of varying experience. This method can provide a new paradigm for pilot neuroergonomics training and early warning of vestibular-visual integration disorders.
Förderung
- National Natural Science Foundation of China
- Key Research; Development Projects of Shaanxi Province
- Key Laboratory of Metrological Optics and Application, State Administration for Market Regulation
Zitieren
APA
Zhang, H., Wang, C., & Wang, P. (2025). Research on Safe Multimodal Detection Method of Pilot Visual Observation Behavior Based on Cognitive State Decoding. Multimodal Technologies and Interaction, 9(10), 103. https://doi.org/10.3390/mti9100103
Vancouver
Zhang H, Wang C, Wang P. Research on Safe Multimodal Detection Method of Pilot Visual Observation Behavior Based on Cognitive State Decoding. Multimodal Technologies and Interaction. 2025;9(10):103. doi:10.3390/mti9100103
BibTeX
@article{Zhang2025,
author = {Zhang, Heming and Wang, Changyuan and Wang, Pengbo},
title = {Research on Safe Multimodal Detection Method of Pilot Visual Observation Behavior Based on Cognitive State Decoding},
journal = {Multimodal Technologies and Interaction},
year = {2025},
volume = {9},
number = {10},
pages = {103},
doi = {10.3390/mti9100103}
}
EUVIMED