EUVIMEDEUVIMED entdecken
Open Access · CC BY-SA 4.0

Deep Learning Based Pain Recognition via Facial Expression Using Feature Fusion of Visual and Thermal Images

Raihan Islamadina, Fitri Arnia, Taufik Fuadi Abidin, Rusdha Muharar, Aulia Syarif Aziz, Khairun Saddami

Journal of Electronics, Electromedical Engineering, and Medical Informatics · 14.07.2026 · 8 (3) · 1163-1183

DOI: 10.35882/jeeemi.v8i3.1766 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of Electronics, Electromedical Engineering, and Medical Informatics, online erschienen am 14.07.2026. Verfasst von Raihan Islamadina und 5 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY-SA 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Objective pain assessment in non-verbal patients remains a significant clinical challenge. While automated facial expression analysis offers a promising solution, existing methods often rely on simple cross-modality concatenation or shallow attention mechanisms, which typically fail to fully capture complex, interdependent cross-modal dynamics. To address this limitation, this study proposes a deep learning-based multimodal feature fusion framework that combines visual and thermal images to significantly enhance pain detection accuracy. The proposed framework integrates pretrained convolutional neural network backbones, specifically VGGFace, ResNet50, and DEYOLO, with two novel attention modules: Dual Semantic Enhancing Channel Weight Assignment (DECA), and Dual Spatial Enhancing Pixel Weight Assignment (DEPA) to adaptively optimize joint feature representations. For comparison, a hybrid baseline model that processes visual and thermal images separately was also developed, allowing a comparative analysis between fusion-based and non-fusion-based approaches. The model performance was systematically evaluated on the MIntPain dataset, which comprises 20 healthy subjects experiencing five distinct levels of pain intensity. To ensure data independence and prevent identity leakage across the training and testing phases, a strict subject-wise data split protocol was implemented with 15 subjects allocated for training and 5 subjects for testing. Experimental results demonstrate that the proposed multimodal fusion framework achieves superior performance, attaining a peak F1 score of 0.938 using the VGGFace backbone. Furthermore, external validation on the UNBC McMaster Shoulder Pain dataset yields a classification accuracy of 0.872, confirming the strong generalization capability and stability of the framework across unseen subjects. These findings highlight the effectiveness of visual-thermal synergy and the efficacy of the proposed DECA and DEPA modules, showcasing high potential for robust, non-invasive clinical pain-monitoring and assessment applications

Zitieren

APA

Islamadina, R., Arnia, F., Abidin, T. F., Muharar, R., Aziz, A. S., & Saddami, K. (2026). Deep Learning Based Pain Recognition via Facial Expression Using Feature Fusion of Visual and Thermal Images. Journal of Electronics, Electromedical Engineering, and Medical Informatics, 8(3), 1163-1183. https://doi.org/10.35882/jeeemi.v8i3.1766

Vancouver

Islamadina R, Arnia F, Abidin TF, Muharar R, Aziz AS, Saddami K. Deep Learning Based Pain Recognition via Facial Expression Using Feature Fusion of Visual and Thermal Images. Journal of Electronics, Electromedical Engineering, and Medical Informatics. 2026;8(3):1163-1183. doi:10.35882/jeeemi.v8i3.1766

BibTeX
@article{Islamadina2026,
  author = {Islamadina, Raihan and Arnia, Fitri and Abidin, Taufik Fuadi and Muharar, Rusdha and Aziz, Aulia Syarif and Saddami, Khairun},
  title = {Deep Learning Based Pain Recognition via Facial Expression Using Feature Fusion of Visual and Thermal Images},
  journal = {Journal of Electronics, Electromedical Engineering, and Medical Informatics},
  year = {2026},
  volume = {8},
  number = {3},
  pages = {1163--1183},
  doi = {10.35882/jeeemi.v8i3.1766}
}
Kostenlos starten →