Spatiotemporal Aggregation Transformer for Object Detection With Neuromorphic Vision Sensors
IEEE Sensors Journal · 15.06.2024 · 24 (12) · 19397-19406
DOI: 10.1109/jsen.2024.3392973 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Sensors Journal, erschienen am 15.06.2024. Verfasst von Zhaoxuan Guo und 3 weiteren Autor:innen. Die Arbeit wurde bisher 13-mal zitiert (Crossref) und nennt 44 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Foundation of National Key Laboratory of Multispectral Information Intelligent Processing Technology.
Abstract (Originalsprache)
To enhance the accuracy of object detection with event-based neuromorphic vision sensors, a novel event-based detector named spatiotemporal aggregation transformer (STAT) is proposed. First, in order to collect sufficient event information to estimate the problem considered, STAT uses a density-based adaptive sampling (DAS) module to sample continuous event stream into multiple groups adaptively. This module can determine the sampling termination condition by quantifying the velocity and size of objects. Second, STAT integrates a sparse event tensor (SET) to establish compatibility between event stream and traditional vision algorithms. SET maps events to a dense representation by end-to-end fitting the optimal mapping function, mitigating the loss of spatiotemporal information within the event stream. Finally, in order to enhance the features of slowly moving objects, a lightweight and efficient triaxial vision transformer (TVT) is designed for modeling global features and integrating historical motion information. Experimental evaluations on two benchmark datasets show that the performance of STAT achieves a mean average precision (mAP) of 68.2% and 49.9% on the Neuromorphic-Caltech101 (N-Caltech101) dataset and the Gen1 dataset, respectively. These results demonstrate that the detection accuracy of STAT outperforms the state-of-the-art methods by 2.0% on the Gen1 dataset. The code of this project is available at https://github.com/TJU-guozhaoxuan/STAT.
Förderung
- National Natural Science Foundation of China
- Foundation of National Key Laboratory of Multispectral Information Intelligent Processing Technology
Zitieren
APA
Guo, Z., Gao, J., Ma, G., & Xu, J. (2024). Spatiotemporal Aggregation Transformer for Object Detection With Neuromorphic Vision Sensors. IEEE Sensors Journal, 24(12), 19397-19406. https://doi.org/10.1109/jsen.2024.3392973
Vancouver
Guo Z, Gao J, Ma G, Xu J. Spatiotemporal Aggregation Transformer for Object Detection With Neuromorphic Vision Sensors. IEEE Sensors Journal. 2024;24(12):19397-19406. doi:10.1109/jsen.2024.3392973
BibTeX
@article{Guo2024,
author = {Guo, Zhaoxuan and Gao, Jiandong and Ma, Guangyuan and Xu, Jiangtao},
title = {Spatiotemporal Aggregation Transformer for Object Detection With Neuromorphic Vision Sensors},
journal = {IEEE Sensors Journal},
year = {2024},
volume = {24},
number = {12},
pages = {19397--19406},
doi = {10.1109/jsen.2024.3392973}
}
EUVIMED