EUVIMEDEUVIMED entdecken
Crossref

Gate-Shift-Fuse for Video Action Recognition

Swathikiran Sudhakaran, Sergio Escalera, Oswald Lanz

IEEE Transactions on Pattern Analysis and Machine Intelligence · 01.09.2023 · 45 (9) · 10913-10928

DOI: 10.1109/tpami.2023.3268134 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, erschienen am 01.09.2023. Verfasst von Swathikiran Sudhakaran und 2 weiteren Autor:innen. Die Arbeit wurde bisher 37-mal zitiert (Crossref) und nennt 82 Literaturangaben. Gefördert von: Spanish projects, Libera Università di Bolzano, Amazon.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Convolutional Neural Networks are the de facto models for image recognition. However 3D CNNs, the straight forward extension of 2D CNNs for video recognition, have not achieved the same success on standard action recognition benchmarks. One of the main reasons for this reduced performance of 3D CNNs is the increased computational complexity requiring large scale annotated datasets to train them in scale. 3D kernel factorization approaches have been proposed to reduce the complexity of 3D CNNs. Existing kernel factorization approaches follow hand-designed and hard-wired techniques. In this paper we propose Gate-Shift-Fuse (GSF), a novel spatio-temporal feature extraction module which controls interactions in spatio-temporal decomposition and learns to adaptively route features through time and combine them in a data dependent manner. GSF leverages grouped spatial gating to decompose input tensor and channel weighting to fuse the decomposed tensors. GSF can be inserted into existing 2D CNNs to convert them into an efficient and high performing spatio-temporal feature extractor, with negligible parameter and compute overhead. We perform an extensive analysis of GSF using two popular 2D CNN families and achieve state-of-the-art or competitive performance on five standard action recognition benchmarks.

Förderung

  • Spanish projects
  • Libera Università di Bolzano
  • Amazon
  • ICREA Academia Programme
  • NVIDIA AI Technology Centre
  • EMEA
  • CINECA Award

Zitieren

APA

Sudhakaran, S., Escalera, S., & Lanz, O. (2023). Gate-Shift-Fuse for Video Action Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(9), 10913-10928. https://doi.org/10.1109/tpami.2023.3268134

Vancouver

Sudhakaran S, Escalera S, Lanz O. Gate-Shift-Fuse for Video Action Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2023;45(9):10913-10928. doi:10.1109/tpami.2023.3268134

BibTeX
@article{Sudhakaran2023,
  author = {Sudhakaran, Swathikiran and Escalera, Sergio and Lanz, Oswald},
  title = {Gate-Shift-Fuse for Video Action Recognition},
  journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year = {2023},
  volume = {45},
  number = {9},
  pages = {10913--10928},
  doi = {10.1109/tpami.2023.3268134}
}
Kostenlos starten →