EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC 4.0

Hybrid Attention Spike Transformer

Xiongfei Fan, Hong Zhang, Yu Zhang

IET Cyber-Systems and Robotics · 2025-01 · 7 (1)

DOI: 10.1049/csy2.70010 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IET Cyber-Systems and Robotics, online erschienen am 10.04.2025. Verfasst von Xiongfei Fan und 2 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 47 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Spike transformers cannot be pretrained due to objective factors such as lack of datasets and memory constraints, which results in a significant performance gap compared to pretrained artificial neural networks (ANNs), thereby hindering their practical applicability. To address this issue, we propose a hybrid attention spike transformer that utilises self‐attention with compound tokens and channel attention‐based token processing to better capture the inductive biases of the data. We also add convolution in patch splitting and feedforward networks, which not only provides local information but also leverages the translation invariance and locality of convolutions to help the model converge. Experiments on static datasets and neuromorphic datasets demonstrate that our method achieves state‐of‐the‐art performance in the spiking neural networks (SNNs) field. Notably, we achieve a top‐1 accuracy of 80.59% on CIFAR‐100 with only 4 time steps. As far as we know, it is the first exploration of the spike transformer with multiattention fusion, achieving outstanding effectiveness.

Zitieren

APA

Fan, X., Zhang, H., & Zhang, Y. (2025). Hybrid Attention Spike Transformer. IET Cyber-Systems and Robotics, 7(1). https://doi.org/10.1049/csy2.70010

Vancouver

Fan X, Zhang H, Zhang Y. Hybrid Attention Spike Transformer. IET Cyber-Systems and Robotics. 2025;7(1). doi:10.1049/csy2.70010

BibTeX
@article{Fan2025,
  author = {Fan, Xiongfei and Zhang, Hong and Zhang, Yu},
  title = {Hybrid Attention Spike Transformer},
  journal = {IET Cyber-Systems and Robotics},
  year = {2025},
  volume = {7},
  number = {1},
  doi = {10.1049/csy2.70010}
}
Infokarte | EUVIMED

Artikeldaten derzeit nicht verfügbar.

Recherche