EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Confidence-Aware Gated Multimodal Fusion for Robust Temporal Action Localization in Occluded Environments

Masato Takami iD, Tomohiro Fukuda iD

Sensors · 16.04.2026 · 26 (8) · 2454

DOI: 10.3390/s26082454 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Sensors, online erschienen am 16.04.2026. Verfasst von Masato Takami und 1 weiteren Autor:innen. Beteiligte Einrichtung: Division of Sustainable Energy and Environmental Engineering, Graduate School of Engineering, The University of Osaka, 2-1 Yamadaoka, Suita 565-0871,…. Die Arbeit nennt 51 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Japan Society for the Promotion of Science.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

In industrial environments, robust Temporal Action Localization (TAL) is essential; however, frequent occlusions often compromise the reliability of skeletal data, leading to negative transfer in multimodal fusion. To address this challenge, we propose a Gated Skeleton Refinement Module (Gated SRM), a universal front-end preprocessing module that explicitly incorporates OpenPose confidence scores into the network architecture. By applying these scores as a logarithmic bias within a self-attention mechanism, our method achieves soft suppression—dynamically attenuating the attention weights assigned to unreliable joints—before adaptively fusing the refined skeletal features with RGB representations through a learnable gating network. Extensive experiments on the heavily occluded IKEA ASM dataset demonstrate that our approach effectively prevents the catastrophic accuracy degradation typical of naive and established multimodal fusion strategies, improving the mean Average Precision (mAP) to 21.77%, maintaining parity with the RGB-only baseline while demonstrating superior robustness. Furthermore, the system maintains a practical end-to-end inference speed of approximately 9.2 frames per second (FPS), which is sufficient for monitoring macro-level industrial workflows. By prioritizing confidence-based data selection over data restoration, this sensor-metadata-driven architecture offers a robust and principled approach acting as a critical fail-safe and safety-net for real-world action recognition under occlusion.

Schlagwörter

Stichwörter der Autor:innen

multimodal learningocclusion handlingpose estimationrobustnesstemporal action localization

Autor:innen und Institutionen

  • Division of Sustainable Energy and Environmental Engineering, Graduate School of Engineering, The University of Osaka, 2-1 Yamadaoka, Suita 565-0871, Osaka, Japan.

ORCID-Profile: Masato Takami, Tomohiro Fukuda

Förderung

  • Japan Society for the Promotion of Science

Zitieren

APA

Takami, M., & Fukuda, T. (2026). Confidence-Aware Gated Multimodal Fusion for Robust Temporal Action Localization in Occluded Environments. Sensors, 26(8), 2454. https://doi.org/10.3390/s26082454

Vancouver

Takami M, Fukuda T. Confidence-Aware Gated Multimodal Fusion for Robust Temporal Action Localization in Occluded Environments. Sensors. 2026;26(8):2454. doi:10.3390/s26082454

BibTeX
@article{Takami2026,
  author = {Takami, Masato and Fukuda, Tomohiro},
  title = {Confidence-Aware Gated Multimodal Fusion for Robust Temporal Action Localization in Occluded Environments},
  journal = {Sensors},
  year = {2026},
  volume = {26},
  number = {8},
  pages = {2454},
  doi = {10.3390/s26082454}
}
Kostenlos starten →