EUVIMEDEUVIMED entdecken
Crossref

Point Spatio-Temporal Transformer Networks for Point Cloud Video Modeling

Hehe Fan, Yi Yang, Mohan Kankanhalli

IEEE Transactions on Pattern Analysis and Machine Intelligence · 01.02.2023 · 45 (2) · 2181-2192

DOI: 10.1109/tpami.2022.3161735 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, erschienen am 01.02.2023. Verfasst von Hehe Fan und 2 weiteren Autor:innen. Die Arbeit wurde bisher 102-mal zitiert (Crossref) und nennt 60 Literaturangaben. Gefördert von: Agency for Science, Technology and Research.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Due to the inherent unorderliness and irregularity of point cloud, points emerge inconsistently across different frames in a point cloud video. To capture the dynamics in point cloud videos, tracking points and limiting temporal modeling range are usually employed to preserve spatio-temporal structure. However, as points may flow in and out across frames, computing accurate point trajectories is extremely difficult, especially for long videos. Moreover, when points move fast, even in a small temporal window, points may still escape from a region. Besides, using the same temporal range for different motions may not accurately capture the temporal structure. In this paper, we propose a Point Spatio-Temporal Transformer (PST-Transformer). To preserve the spatio-temporal structure, PST-Transformer adaptively searches related or similar points across the entire video by performing self-attention on point features. Moreover, our PST-Transformer is equipped with an ability to encode spatio-temporal structure. Because point coordinates are irregular and unordered but point timestamps exhibit regularities and order, the spatio-temporal encoding is decoupled to reduce the impact of the spatial irregularity on the temporal modeling. By properly preserving and encoding spatio-temporal structure, our PST-Transformer effectively models point cloud videos and shows superior performance on 3D action recognition and 4D semantic segmentation.

Förderung

  • Agency for Science, Technology and Research

Zitieren

APA

Fan, H., Yang, Y., & Kankanhalli, M. (2023). Point Spatio-Temporal Transformer Networks for Point Cloud Video Modeling. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(2), 2181-2192. https://doi.org/10.1109/tpami.2022.3161735

Vancouver

Fan H, Yang Y, Kankanhalli M. Point Spatio-Temporal Transformer Networks for Point Cloud Video Modeling. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2023;45(2):2181-2192. doi:10.1109/tpami.2022.3161735

BibTeX
@article{Fan2023,
  author = {Fan, Hehe and Yang, Yi and Kankanhalli, Mohan},
  title = {Point Spatio-Temporal Transformer Networks for Point Cloud Video Modeling},
  journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year = {2023},
  volume = {45},
  number = {2},
  pages = {2181--2192},
  doi = {10.1109/tpami.2022.3161735}
}
Kostenlos starten →