EUVIMEDEUVIMED entdecken
Crossref

Spatiotemporal Aggregation Transformer for Object Detection With Neuromorphic Vision Sensors

Zhaoxuan Guo, Jiandong Gao, Guangyuan Ma, Jiangtao Xu

IEEE Sensors Journal · 15.06.2024 · 24 (12) · 19397-19406

DOI: 10.1109/jsen.2024.3392973 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Sensors Journal, erschienen am 15.06.2024. Verfasst von Zhaoxuan Guo und 3 weiteren Autor:innen. Die Arbeit wurde bisher 13-mal zitiert (Crossref) und nennt 44 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Foundation of National Key Laboratory of Multispectral Information Intelligent Processing Technology.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

To enhance the accuracy of object detection with event-based neuromorphic vision sensors, a novel event-based detector named spatiotemporal aggregation transformer (STAT) is proposed. First, in order to collect sufficient event information to estimate the problem considered, STAT uses a density-based adaptive sampling (DAS) module to sample continuous event stream into multiple groups adaptively. This module can determine the sampling termination condition by quantifying the velocity and size of objects. Second, STAT integrates a sparse event tensor (SET) to establish compatibility between event stream and traditional vision algorithms. SET maps events to a dense representation by end-to-end fitting the optimal mapping function, mitigating the loss of spatiotemporal information within the event stream. Finally, in order to enhance the features of slowly moving objects, a lightweight and efficient triaxial vision transformer (TVT) is designed for modeling global features and integrating historical motion information. Experimental evaluations on two benchmark datasets show that the performance of STAT achieves a mean average precision (mAP) of 68.2% and 49.9% on the Neuromorphic-Caltech101 (N-Caltech101) dataset and the Gen1 dataset, respectively. These results demonstrate that the detection accuracy of STAT outperforms the state-of-the-art methods by 2.0% on the Gen1 dataset. The code of this project is available at https://github.com/TJU-guozhaoxuan/STAT.

Förderung

  • National Natural Science Foundation of China
  • Foundation of National Key Laboratory of Multispectral Information Intelligent Processing Technology

Zitieren

APA

Guo, Z., Gao, J., Ma, G., & Xu, J. (2024). Spatiotemporal Aggregation Transformer for Object Detection With Neuromorphic Vision Sensors. IEEE Sensors Journal, 24(12), 19397-19406. https://doi.org/10.1109/jsen.2024.3392973

Vancouver

Guo Z, Gao J, Ma G, Xu J. Spatiotemporal Aggregation Transformer for Object Detection With Neuromorphic Vision Sensors. IEEE Sensors Journal. 2024;24(12):19397-19406. doi:10.1109/jsen.2024.3392973

BibTeX
@article{Guo2024,
  author = {Guo, Zhaoxuan and Gao, Jiandong and Ma, Guangyuan and Xu, Jiangtao},
  title = {Spatiotemporal Aggregation Transformer for Object Detection With Neuromorphic Vision Sensors},
  journal = {IEEE Sensors Journal},
  year = {2024},
  volume = {24},
  number = {12},
  pages = {19397--19406},
  doi = {10.1109/jsen.2024.3392973}
}
Kostenlos starten →