EUVIMEDEUVIMED entdecken
Crossref

Self-Supervised Event Data Pre-Training via Coarse-to-Fine Temporal Reconstruction for Spiking Neural Networks

Qirui Huang, Jianhao Li, Chenchen Yan, Pengjiang Qian, Guoqi Li, Jiangbo Qian

IEEE Transactions on Circuits and Systems for Video Technology · 2026-11 · 36 (11) · 16947-16962

DOI: 10.1109/tcsvt.2026.3717439 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Circuits and Systems for Video Technology, 2026-11. Verfasst von Qirui Huang und 5 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 80 Literaturangaben. Gefördert von: Zhejiang Provincial Natural Science Foundation of China, NSF, China,, Ningbo Major Science and Technology Project of China.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Event data exhibit high temporal resolution, low latency, and spatial sparsity characteristics that are inherently compatible with the energy-efficient computation advantages of spiking neural networks through their unique spike-driven mechanisms. Existing methods typically first convert event streams into frames and then extract features using convolutional operations. However, in the conversion process, a significant amount of fine-grained temporal information is discarded from the event stream, which hinders recognition capability. To address this issue, we propose a frame-point spiking transformer (FPSformer), which introduces event points containing fine-grained temporal information into event frames and employs a spiking cross-attention mechanism to fuse these features, thereby enhancing performance. To further exploit the temporal information in event data and improve the performance of FPSformer, we investigate the distinct characteristics of event data and propose a novel event data pre-training method called the masked autoencoder for event data (EVMAE). While conventional masked autoencoders focus on learning spatial information by reconstructing the original input image rather than learning temporal information, our EVMAE reconstructs finely sliced event frames from coarsely sliced inputs, encouraging the model to focus more on learning temporal representations that are crucial for event-based action recognition. To our knowledge, EVMAE is the first masked modeling approach specifically designed for event-based action recognition. We conduct extensive experiments on multiple neuromorphic datasets to validate the effectiveness of our approach. Compared with existing methods, EVMAE demonstrates superior performance on several downstream tasks.

Förderung

  • Zhejiang Provincial Natural Science Foundation of China
  • NSF, China,
  • Ningbo Major Science and Technology Project of China
  • K

Zitieren

APA

Huang, Q., Li, J., Yan, C., Qian, P., Li, G., & Qian, J. (2026). Self-Supervised Event Data Pre-Training via Coarse-to-Fine Temporal Reconstruction for Spiking Neural Networks. IEEE Transactions on Circuits and Systems for Video Technology, 36(11), 16947-16962. https://doi.org/10.1109/tcsvt.2026.3717439

Vancouver

Huang Q, Li J, Yan C, Qian P, Li G, Qian J. Self-Supervised Event Data Pre-Training via Coarse-to-Fine Temporal Reconstruction for Spiking Neural Networks. IEEE Transactions on Circuits and Systems for Video Technology. 2026;36(11):16947-16962. doi:10.1109/tcsvt.2026.3717439

BibTeX
@article{Huang2026,
  author = {Huang, Qirui and Li, Jianhao and Yan, Chenchen and Qian, Pengjiang and Li, Guoqi and Qian, Jiangbo},
  title = {Self-Supervised Event Data Pre-Training via Coarse-to-Fine Temporal Reconstruction for Spiking Neural Networks},
  journal = {IEEE Transactions on Circuits and Systems for Video Technology},
  year = {2026},
  volume = {36},
  number = {11},
  pages = {16947--16962},
  doi = {10.1109/tcsvt.2026.3717439}
}
Kostenlos starten →