EUVIMEDEUVIMED entdecken
Crossref

Action Recognition and Benchmark Using Event Cameras

Yue Gao, Jiaxuan Lu, Siqi Li, Nan Ma, Shaoyi Du, Yipeng Li, Qionghai Dai

IEEE Transactions on Pattern Analysis and Machine Intelligence · 2023-12 · 45 (12) · 14081-14097

DOI: 10.1109/tpami.2023.3300741 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023-12. Verfasst von Yue Gao und 6 weiteren Autor:innen. Die Arbeit wurde bisher 59-mal zitiert (Crossref) und nennt 58 Literaturangaben. Gefördert von: National Key Research and Development Program of China.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Recent years have witnessed remarkable achievements in video-based action recognition. Apart from traditional frame-based cameras, event cameras are bio-inspired vision sensors that only record pixel-wise brightness changes rather than the brightness value. However, little effort has been made in event-based action recognition, and large-scale public datasets are also nearly unavailable. In this paper, we propose an event-based action recognition framework called EV-ACT. The Learnable Multi-Fused Representation (LMFR) is first proposed to integrate multiple event information in a learnable manner. The LMFR with dual temporal granularity is fed into the event-based slow-fast network for the fusion of appearance and motion features. A spatial-temporal attention mechanism is introduced to further enhance the learning capability of action recognition. To prompt research in this direction, we have collected the largest event-based action recognition benchmark named THUE-ACT-50 and the accompanying THUE-ACT-50-CHL dataset under challenging environments, including a total of over 12,830 recordings from 50 action categories, which is over 4 times the size of the previous largest dataset. Experimental results show that our proposed framework could achieve improvements of over 14.5%, 7.6%, 11.2%, and 7.4% compared to previous works on four benchmarks. We have also deployed our proposed EV-ACT framework on a mobile platform to validate its practicality and efficiency.

Förderung

  • National Key Research and Development Program of China

Zitieren

APA

Gao, Y., Lu, J., Li, S., Ma, N., Du, S., Li, Y., & Dai, Q. (2023). Action Recognition and Benchmark Using Event Cameras. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(12), 14081-14097. https://doi.org/10.1109/tpami.2023.3300741

Vancouver

Gao Y, Lu J, Li S, Ma N, Du S, Li Y, et al. Action Recognition and Benchmark Using Event Cameras. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2023;45(12):14081-14097. doi:10.1109/tpami.2023.3300741

BibTeX
@article{Gao2023,
  author = {Gao, Yue and Lu, Jiaxuan and Li, Siqi and Ma, Nan and Du, Shaoyi and Li, Yipeng and Dai, Qionghai},
  title = {Action Recognition and Benchmark Using Event Cameras},
  journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year = {2023},
  volume = {45},
  number = {12},
  pages = {14081--14097},
  doi = {10.1109/tpami.2023.3300741}
}
Kostenlos starten →