EUVIMEDEUVIMED entdecken
Crossref

Spike2Rep: Learning Compact Spatiotemporal Representations for Recognition With High-Speed Spike-Based Image Sensor

Feifan Zhang, Jing Gao, Quanmin Chen, Jiangtao Xu

IEEE Sensors Journal · 2026-08 · 26 (16) · 25171-25180

DOI: 10.1109/jsen.2026.3708153 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Sensors Journal, 2026-08. Verfasst von Feifan Zhang und 3 weiteren Autor:innen. Die Arbeit nennt 45 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Emerging Frontiers Cultivation Program of Tianjin University Interdisciplinary Center.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

High-speed spike-based image sensors have shown great potential for high-speed object recognition due to their ultrahigh temporal resolution and static scene retention. To fully exploit this hardware capability, converting spike streams into structured representations suitable for deep neural networks remains a key challenge. Notably, object categories are temporally invariant, while spike streams exhibit rich temporal dynamics, suggesting exploitable spatiotemporal redundancy. In this work, Spike2Rep is proposed to exploit the intrinsic spatiotemporal redundancy in spike streams for a compact representation. In particular, the temporal dynamics are treated as motion cues to reveal spatial importance, and temporal redundancy is used for adaptive temporal aggregation. To implement this, the discrete wavelet transform (DWT) is applied to obtain a compact time–frequency representation from spike streams. Subsequently, the temporal difference is employed to capture motion cues and generate spatial importance maps, which are added to enhance features. Furthermore, temporal and channel weights are predicted via a linear layer, which is used for adaptive modulation. Finally, the temporal averaging is adopted to produce a compact spatiotemporal representation. On the S-CALTECH dataset, the proposed method achieves up to 81.27% top-1 accuracy using 2.2 ms of spike streams and 87.73% with 6.4 ms, surpassing the prior state-of-the-art (SOTA) methods on this dataset.

Förderung

  • National Natural Science Foundation of China
  • Emerging Frontiers Cultivation Program of Tianjin University Interdisciplinary Center

Zitieren

APA

Zhang, F., Gao, J., Chen, Q., & Xu, J. (2026). Spike2Rep: Learning Compact Spatiotemporal Representations for Recognition With High-Speed Spike-Based Image Sensor. IEEE Sensors Journal, 26(16), 25171-25180. https://doi.org/10.1109/jsen.2026.3708153

Vancouver

Zhang F, Gao J, Chen Q, Xu J. Spike2Rep: Learning Compact Spatiotemporal Representations for Recognition With High-Speed Spike-Based Image Sensor. IEEE Sensors Journal. 2026;26(16):25171-25180. doi:10.1109/jsen.2026.3708153

BibTeX
@article{Zhang2026,
  author = {Zhang, Feifan and Gao, Jing and Chen, Quanmin and Xu, Jiangtao},
  title = {Spike2Rep: Learning Compact Spatiotemporal Representations for Recognition With High-Speed Spike-Based Image Sensor},
  journal = {IEEE Sensors Journal},
  year = {2026},
  volume = {26},
  number = {16},
  pages = {25171--25180},
  doi = {10.1109/jsen.2026.3708153}
}
Kostenlos starten →