EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Research on Safe Multimodal Detection Method of Pilot Visual Observation Behavior Based on Cognitive State Decoding

Heming Zhang, Changyuan Wang, Pengbo Wang

Multimodal Technologies and Interaction · 01.10.2025 · 9 (10) · 103

DOI: 10.3390/mti9100103 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Multimodal Technologies and Interaction, online erschienen am 01.10.2025. Verfasst von Heming Zhang und 2 weiteren Autor:innen. Die Arbeit wurde bisher 5-mal zitiert (Crossref) und nennt 42 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Natural Science Foundation of China, Key Research; Development Projects of Shaanxi Province, Key Laboratory of Metrological Optics and Application, State Administration for Market Regulation.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Pilot visual behavior safety assessment is a cross-disciplinary technology that analyzes pilots’ gaze behavior and neurocognitive responses. This paper proposes a multimodal analysis method for pilot visual behavior safety, specifically for cognitive state decoding. This method aims to achieve a quantitative and efficient assessment of pilots’ observational behavior. Addressing the subjective limitations of traditional methods, this paper proposes an observational behavior detection model that integrates facial images to achieve dynamic and quantitative analysis of observational behavior. It addresses the “Midas contact” problem of observational behavior by constructing a cognitive analysis method using multimodal signals. We propose a bidirectional long short-term memory (LSTM) network that matches physiological signal rhythmic features to address the problem of isolated features in multidimensional signals. This method captures the dynamic correlations between multiple physiological behaviors, such as prefrontal theta and chest-abdominal coordination, to decode the cognitive state of pilots’ observational behavior. Finally, the paper uses a decision-level fusion method based on an improved Dempster–Shafer (DS) evidence theory to provide a quantifiable detection strategy for aviation safety standards. This dual-dimensional quantitative assessment system of “visual behavior–neurophysiological cognition” reveals the dynamic correlations between visual behavior and cognitive state among pilots of varying experience. This method can provide a new paradigm for pilot neuroergonomics training and early warning of vestibular-visual integration disorders.

Förderung

  • National Natural Science Foundation of China
  • Key Research; Development Projects of Shaanxi Province
  • Key Laboratory of Metrological Optics and Application, State Administration for Market Regulation

Zitieren

APA

Zhang, H., Wang, C., & Wang, P. (2025). Research on Safe Multimodal Detection Method of Pilot Visual Observation Behavior Based on Cognitive State Decoding. Multimodal Technologies and Interaction, 9(10), 103. https://doi.org/10.3390/mti9100103

Vancouver

Zhang H, Wang C, Wang P. Research on Safe Multimodal Detection Method of Pilot Visual Observation Behavior Based on Cognitive State Decoding. Multimodal Technologies and Interaction. 2025;9(10):103. doi:10.3390/mti9100103

BibTeX
@article{Zhang2025,
  author = {Zhang, Heming and Wang, Changyuan and Wang, Pengbo},
  title = {Research on Safe Multimodal Detection Method of Pilot Visual Observation Behavior Based on Cognitive State Decoding},
  journal = {Multimodal Technologies and Interaction},
  year = {2025},
  volume = {9},
  number = {10},
  pages = {103},
  doi = {10.3390/mti9100103}
}
Kostenlos starten →