Confidence-Aware Gated Multimodal Fusion for Robust Temporal Action Localization in Occluded Environments
Sensors · 16.04.2026 · 26 (8) · 2454
DOI: 10.3390/s26082454 ↗Kurz gefasst
Fachartikel in der Zeitschrift Sensors, online erschienen am 16.04.2026. Verfasst von Masato Takami und 1 weiteren Autor:innen. Beteiligte Einrichtung: Division of Sustainable Energy and Environmental Engineering, Graduate School of Engineering, The University of Osaka, 2-1 Yamadaoka, Suita 565-0871,…. Die Arbeit nennt 51 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Japan Society for the Promotion of Science.
Abstract (Originalsprache)
In industrial environments, robust Temporal Action Localization (TAL) is essential; however, frequent occlusions often compromise the reliability of skeletal data, leading to negative transfer in multimodal fusion. To address this challenge, we propose a Gated Skeleton Refinement Module (Gated SRM), a universal front-end preprocessing module that explicitly incorporates OpenPose confidence scores into the network architecture. By applying these scores as a logarithmic bias within a self-attention mechanism, our method achieves soft suppression—dynamically attenuating the attention weights assigned to unreliable joints—before adaptively fusing the refined skeletal features with RGB representations through a learnable gating network. Extensive experiments on the heavily occluded IKEA ASM dataset demonstrate that our approach effectively prevents the catastrophic accuracy degradation typical of naive and established multimodal fusion strategies, improving the mean Average Precision (mAP) to 21.77%, maintaining parity with the RGB-only baseline while demonstrating superior robustness. Furthermore, the system maintains a practical end-to-end inference speed of approximately 9.2 frames per second (FPS), which is sufficient for monitoring macro-level industrial workflows. By prioritizing confidence-based data selection over data restoration, this sensor-metadata-driven architecture offers a robust and principled approach acting as a critical fail-safe and safety-net for real-world action recognition under occlusion.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Division of Sustainable Energy and Environmental Engineering, Graduate School of Engineering, The University of Osaka, 2-1 Yamadaoka, Suita 565-0871, Osaka, Japan.
ORCID-Profile: Masato Takami, Tomohiro Fukuda
Förderung
- Japan Society for the Promotion of Science
Zitieren
APA
Takami, M., & Fukuda, T. (2026). Confidence-Aware Gated Multimodal Fusion for Robust Temporal Action Localization in Occluded Environments. Sensors, 26(8), 2454. https://doi.org/10.3390/s26082454
Vancouver
Takami M, Fukuda T. Confidence-Aware Gated Multimodal Fusion for Robust Temporal Action Localization in Occluded Environments. Sensors. 2026;26(8):2454. doi:10.3390/s26082454
BibTeX
@article{Takami2026,
author = {Takami, Masato and Fukuda, Tomohiro},
title = {Confidence-Aware Gated Multimodal Fusion for Robust Temporal Action Localization in Occluded Environments},
journal = {Sensors},
year = {2026},
volume = {26},
number = {8},
pages = {2454},
doi = {10.3390/s26082454}
}
EUVIMED