On Fatigue Detection for Air Traffic Controllers Based on Fuzzy Fusion of Multiple Features
Computational and Mathematical Methods in Medicine · 11.10.2022 · 2022 · 1-10
DOI: 10.1155/2022/4911005 ↗Kurz gefasst
Fachartikel in der Zeitschrift Computational and Mathematical Methods in Medicine, erschienen am 11.10.2022. Verfasst von Yi Hu und 6 weiteren Autor:innen. Beteiligte Einrichtung: College of Civil Aviation, Nanjing University of Aeronautics and Astronautics, Nanjing, Jiangsu 210016, China. Hauptthemen laut MEDLINE: Voice. Die Arbeit wurde bisher 14-mal zitiert (Crossref) und nennt 30 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Foundation for Safety and Capacity Development of Civil Aviation Administration of China, National Natural Science Foundation of China.
Abstract (Originalsprache)
Fatigue detection for air traffic controllers is an important yet challenging problem in aviation safety research. Most of the existing methods for this problem are based on facial features. In this paper, we propose an ensemble learning model that combines both facial features and voice features and design a fatigue detection method through multifeature fusion, referred to as Facial and Voice Stacking (FV-Stacking). Specifically, for facial features, we first use OpenCV and Dlib libraries to extract mouth and eye areas and then employ a combination of M-Convolutional Neural Network (M-CNN) and E-Convolutional Neural Network (E-CNN) to determine the state of mouth and eye closure based on five features, i.e., blinking times, average blinking time, average blinking interval, Percentage of Eyelid Closure over the Pupil over Time (PERCLOS), and Frequency of Open Mouth (FOM). For voice features, we extract the Mel-Frequency Cepstral Coefficients (MFCC) features of speech. Such facial features and voice features are fused through a carefully designed stacking model for fatigue detection. Real-life experiments are conducted on 14 air traffic controllers in Southwest Air Traffic Management Bureau of Civil Aviation of China. The results show that the proposed FV-Stacking method achieves a detection accuracy of 97%, while the best accuracy achieved by a single model is 92% and the best accuracy achieved by the state-of-the-art detection methods is 88%.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- College of Civil Aviation, Nanjing University of Aeronautics and Astronautics, Nanjing, Jiangsu 210016, China.
- School of Information Science and Technology, Northwest University, Xi'an, Shaanxi 710127, China.
- Department of Data Science, New Jersey Institute of Technology, Newark, NJ 07102, USA.
- Department of Aviation and Technology, San Jose State University, San Jose, CA 95192, USA.
- Zhongke Haoyin Intelligent Technology Co., Ltd., Hefei, Anhui 230088, China.
ORCID-Profile: Yi Hu, Zhuo Liu, Aiqin Hou, Chase Wu, Wenbin Wei, Yanjun Wang, Min Liu
Förderung
- Foundation for Safety and Capacity Development of Civil Aviation Administration of China
- National Natural Science Foundation of China
Zitieren
APA
Hu, Y., Liu, Z., Hou, A., Wu, C., Wei, W., Wang, Y., & Liu, M. (2022). On Fatigue Detection for Air Traffic Controllers Based on Fuzzy Fusion of Multiple Features. Computational and Mathematical Methods in Medicine, 2022, 1-10. https://doi.org/10.1155/2022/4911005
Vancouver
Hu Y, Liu Z, Hou A, Wu C, Wei W, Wang Y, et al. On Fatigue Detection for Air Traffic Controllers Based on Fuzzy Fusion of Multiple Features. Computational and Mathematical Methods in Medicine. 2022;2022:1-10. doi:10.1155/2022/4911005
BibTeX
@article{Hu2022,
author = {Hu, Yi and Liu, Zhuo and Hou, Aiqin and Wu, Chase and Wei, Wenbin and Wang, Yanjun and Liu, Min},
title = {On Fatigue Detection for Air Traffic Controllers Based on Fuzzy Fusion of Multiple Features},
journal = {Computational and Mathematical Methods in Medicine},
year = {2022},
volume = {2022},
pages = {1--10},
doi = {10.1155/2022/4911005}
}
EUVIMED