EUVIMEDEUVIMED entdecken
ValidierungsstudieOpen Access · CC BY 4.0

Deep learning-based bimodal speech and facial expression recognition of miners’ unsafe emotions

Ying Lu iD, Zihao Zhao iD, Liang Yan, Xinyv Shi

PLOS One · 15.05.2026 · 21 (5) · e0348906

DOI: 10.1371/journal.pone.0348906 ↗

Kurz gefasst

Validierungsstudie in der Zeitschrift PLOS One, online erschienen am 15.05.2026. Verfasst von Ying Lu und 3 weiteren Autor:innen. Beteiligte Einrichtung: School of Resource and Environmental Engineering, Wuhan University of Science and Technology, Wuhan, Hubei, China. Hauptthemen laut MEDLINE: Deep Learning, Miners, Safety Management, Automated Facial Recognition. Die Arbeit nennt 44 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Special Project of Safety Production of the Hubei Emergency Management Department.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Under the influence of unsafe emotions, miners’ ability to perceive risks is hindered, which can easily lead to decision-making errors and safety accidents. To recognize unsafe emotions exhibited by miners during operations, this study proposes a deep learning-based bimodal framework that integrates speech and facial expression features. A convolutional neural network (CNN) combined with a bidirectional long short-term memory (Bi-LSTM) network is employed to model local spectral patterns and temporal dependencies in speech signals, and ShuffleNet-V2 is used to capture deep facial features. In addition, three feature enhancement strategies are proposed to improve the generalization ability of the model. By constructing a dataset containing five categories of miners’ unsafe emotions for network training, the model achieves a mean recognition accuracy of 85.56%. Furthermore, we conducted a preliminary field test of the bimodal model in a real mining environment. The results provide preliminary evidence of its potential applicability in real-world mining conditions.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Deep LearningMinersSafety ManagementAutomated Facial RecognitionSpeech Recognition SoftwareEmotionsLong Short Term MemoryConvolutional Neural NetworksSpeechMiningHumansAnxietyIrritable Mood

Autor:innen und Institutionen

  • School of Resource and Environmental Engineering, Wuhan University of Science and Technology, Wuhan, Hubei, China.
  • Hubei Industrial Safety Engineering Technology Research Center, Wuhan, Hubei, China.
  • School of Management, Fudan University, Shanghai, China.
  • Hubei Chemical Safety Association, Wuhan, Hubei, China.

ORCID-Profile: Ying Lu, Zihao Zhao

Förderung

  • Special Project of Safety Production of the Hubei Emergency Management Department

Zitieren

APA

Lu, Y., Zhao, Z., Yan, L., & Shi, X. (2026). Deep learning-based bimodal speech and facial expression recognition of miners’ unsafe emotions. PLOS One, 21(5), e0348906. https://doi.org/10.1371/journal.pone.0348906

Vancouver

Lu Y, Zhao Z, Yan L, Shi X. Deep learning-based bimodal speech and facial expression recognition of miners’ unsafe emotions. PLOS One. 2026;21(5):e0348906. doi:10.1371/journal.pone.0348906

BibTeX
@article{Lu2026,
  author = {Lu, Ying and Zhao, Zihao and Yan, Liang and Shi, Xinyv},
  title = {Deep learning-based bimodal speech and facial expression recognition of miners’ unsafe emotions},
  journal = {PLOS One},
  year = {2026},
  volume = {21},
  number = {5},
  pages = {e0348906},
  doi = {10.1371/journal.pone.0348906}
}
Kostenlos starten →