Symbiotic Attention for Egocentric Action Recognition With Object-Centric Alignment
IEEE Transactions on Pattern Analysis and Machine Intelligence · 01.06.2023 · 45 (6) · 6605-6617
DOI: 10.1109/tpami.2020.3015894 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, erschienen am 01.06.2023. Verfasst von Xiaohan Wang und 3 weiteren Autor:innen. Die Arbeit wurde bisher 66-mal zitiert (Crossref) und nennt 58 Literaturangaben.
Abstract (Originalsprache)
In this paper, we propose to tackle egocentric action recognition by suppressing background distractors and enhancing action-relevant interactions. The existing approaches usually utilize two independent branches to recognize egocentric actions, i.e., a verb branch and a noun branch. However, the mechanism to suppress distracting objects and exploit local human-object correlations is missing. To this end, we introduce two extra sources of information, i.e., the candidate objects spatial location and their discriminative features, to enable concentration on the occurring interactions. We design a Symbiotic Attention with Object-centric feature Alignment framework (SAOA) to provide meticulous reasoning between the actor and the environment. First, we introduce an object-centric feature alignment method to inject the local object features to the verb branch and noun branch. Second, we propose a symbiotic attention mechanism to encourage the mutual interaction between the two branches and select the most action-relevant candidates for classification. The framework benefits from the communication among the verb branch, the noun branch, and the local object information. Experiments based on different backbones and modalities demonstrate the effectiveness of our method. Notably, our framework achieves the state-of-the-art on the largest egocentric video dataset.
Zitieren
APA
Wang, X., Zhu, L., Wu, Y., & Yang, Y. (2023). Symbiotic Attention for Egocentric Action Recognition With Object-Centric Alignment. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(6), 6605-6617. https://doi.org/10.1109/tpami.2020.3015894
Vancouver
Wang X, Zhu L, Wu Y, Yang Y. Symbiotic Attention for Egocentric Action Recognition With Object-Centric Alignment. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2023;45(6):6605-6617. doi:10.1109/tpami.2020.3015894
BibTeX
@article{Wang2023,
author = {Wang, Xiaohan and Zhu, Linchao and Wu, Yu and Yang, Yi},
title = {Symbiotic Attention for Egocentric Action Recognition With Object-Centric Alignment},
journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
year = {2023},
volume = {45},
number = {6},
pages = {6605--6617},
doi = {10.1109/tpami.2020.3015894}
}
EUVIMED