Self-Supervised Event Data Pre-Training via Coarse-to-Fine Temporal Reconstruction for Spiking Neural Networks
IEEE Transactions on Circuits and Systems for Video Technology · 2026-11 · 36 (11) · 16947-16962
DOI: 10.1109/tcsvt.2026.3717439 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Circuits and Systems for Video Technology, 2026-11. Verfasst von Qirui Huang und 5 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 80 Literaturangaben. Gefördert von: Zhejiang Provincial Natural Science Foundation of China, NSF, China,, Ningbo Major Science and Technology Project of China.
Abstract (Originalsprache)
Event data exhibit high temporal resolution, low latency, and spatial sparsity characteristics that are inherently compatible with the energy-efficient computation advantages of spiking neural networks through their unique spike-driven mechanisms. Existing methods typically first convert event streams into frames and then extract features using convolutional operations. However, in the conversion process, a significant amount of fine-grained temporal information is discarded from the event stream, which hinders recognition capability. To address this issue, we propose a frame-point spiking transformer (FPSformer), which introduces event points containing fine-grained temporal information into event frames and employs a spiking cross-attention mechanism to fuse these features, thereby enhancing performance. To further exploit the temporal information in event data and improve the performance of FPSformer, we investigate the distinct characteristics of event data and propose a novel event data pre-training method called the masked autoencoder for event data (EVMAE). While conventional masked autoencoders focus on learning spatial information by reconstructing the original input image rather than learning temporal information, our EVMAE reconstructs finely sliced event frames from coarsely sliced inputs, encouraging the model to focus more on learning temporal representations that are crucial for event-based action recognition. To our knowledge, EVMAE is the first masked modeling approach specifically designed for event-based action recognition. We conduct extensive experiments on multiple neuromorphic datasets to validate the effectiveness of our approach. Compared with existing methods, EVMAE demonstrates superior performance on several downstream tasks.
Förderung
- Zhejiang Provincial Natural Science Foundation of China
- NSF, China,
- Ningbo Major Science and Technology Project of China
- K
Zitieren
APA
Huang, Q., Li, J., Yan, C., Qian, P., Li, G., & Qian, J. (2026). Self-Supervised Event Data Pre-Training via Coarse-to-Fine Temporal Reconstruction for Spiking Neural Networks. IEEE Transactions on Circuits and Systems for Video Technology, 36(11), 16947-16962. https://doi.org/10.1109/tcsvt.2026.3717439
Vancouver
Huang Q, Li J, Yan C, Qian P, Li G, Qian J. Self-Supervised Event Data Pre-Training via Coarse-to-Fine Temporal Reconstruction for Spiking Neural Networks. IEEE Transactions on Circuits and Systems for Video Technology. 2026;36(11):16947-16962. doi:10.1109/tcsvt.2026.3717439
BibTeX
@article{Huang2026,
author = {Huang, Qirui and Li, Jianhao and Yan, Chenchen and Qian, Pengjiang and Li, Guoqi and Qian, Jiangbo},
title = {Self-Supervised Event Data Pre-Training via Coarse-to-Fine Temporal Reconstruction for Spiking Neural Networks},
journal = {IEEE Transactions on Circuits and Systems for Video Technology},
year = {2026},
volume = {36},
number = {11},
pages = {16947--16962},
doi = {10.1109/tcsvt.2026.3717439}
}
EUVIMED