EUVIMEDEUVIMED entdecken
Crossref

DVANet: Disentangling View and Action Features for Multi-View Action Recognition

Nyle Siddiqui, Praveen Tirupattur, Mubarak Shah

Proceedings of the AAAI Conference on Artificial Intelligence · 24.03.2024 · 38 (5) · 4873-4881

DOI: 10.1609/aaai.v38i5.28290 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 24.03.2024. Verfasst von Nyle Siddiqui und 2 weiteren Autor:innen. Die Arbeit wurde bisher 35-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

In this work, we present a novel approach to multi-view action recognition where we guide learned action representations to be separated from view-relevant information in a video. When trying to classify action instances captured from multiple viewpoints, there is a higher degree of difficulty due to the difference in background, occlusion, and visibility of the captured action from different camera angles. To tackle the various problems introduced in multi-view action recognition, we propose a novel configuration of learnable transformer decoder queries, in conjunction with two supervised contrastive losses, to enforce the learning of action features that are robust to shifts in viewpoints. Our disentangled feature learning occurs in two stages: the transformer decoder uses separate queries to separately learn action and view information, which are then further disentangled using our two contrastive losses. We show that our model and method of training significantly outperforms all other uni-modal models on four multi-view action recognition datasets: NTU RGB+D, NTU RGB+D 120, PKU-MMD, and N-UCLA. Compared to previous RGB works, we see maximal improvements of 1.5%, 4.8%, 2.2%, and 4.8% on each dataset, respectively. Our code can be found here: https://github.com/NyleSiddiqui/MultiView_Actions

Zitieren

APA

Siddiqui, N., Tirupattur, P., & Shah, M. (2024). DVANet: Disentangling View and Action Features for Multi-View Action Recognition. Proceedings of the AAAI Conference on Artificial Intelligence, 38(5), 4873-4881. https://doi.org/10.1609/aaai.v38i5.28290

Vancouver

Siddiqui N, Tirupattur P, Shah M. DVANet: Disentangling View and Action Features for Multi-View Action Recognition. Proceedings of the AAAI Conference on Artificial Intelligence. 2024;38(5):4873-4881. doi:10.1609/aaai.v38i5.28290

BibTeX
@article{Siddiqui2024,
  author = {Siddiqui, Nyle and Tirupattur, Praveen and Shah, Mubarak},
  title = {DVANet: Disentangling View and Action Features for Multi-View Action Recognition},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2024},
  volume = {38},
  number = {5},
  pages = {4873--4881},
  doi = {10.1609/aaai.v38i5.28290}
}
Kostenlos starten →