EUVIMEDEUVIMED entdecken
Crossref

Revisiting Attention for Multivariate Time Series Forecasting

Haixiang Wu

Proceedings of the AAAI Conference on Artificial Intelligence · 11.04.2025 · 39 (20) · 21528-21535

DOI: 10.1609/aaai.v39i20.35455 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 11.04.2025. Verfasst von Haixiang Wu. Die Arbeit wurde bisher 15-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Current Transformer methods for Multivariate Time-Series Forecasting (MTSF) are all based on the conventional attention mechanism. They involve sequence embedding and performing a linear projection for Q, K, and V, and then computing attention within this latent space. We have not yet delved into the attention mechanism to explore whether such a mapping space is optimal for MTSF. To investigate this issue, we first propose Frequency Spectrum attention (FSatten), a novel attention mechanism based on the frequency domain space. It employs the Fourier transform for embedding and introduces Multi-head Spectrum Scaling (MSS) to replace the conventional linear mapping for Q and K. FSatten can accurately capture the periodic dependencies between sequences and outperform the conventional attention, without necessitating changes to mainstream architectures. We further design a more general method dubbed Scaled Orthogonal attention (SOatten). We propose an orthogonal embedding and a Head-Coupling Convolution (HCC) based on the neighboring similarity bias to guide the model in learning comprehensive dependency patterns. Experiments show that FSatten and SOatten surpass the SOTA which uses conventional attention, making it a good alternative as a basic attention mechanism for MTSF.

Zitieren

APA

Wu, H. (2025). Revisiting Attention for Multivariate Time Series Forecasting. Proceedings of the AAAI Conference on Artificial Intelligence, 39(20), 21528-21535. https://doi.org/10.1609/aaai.v39i20.35455

Vancouver

Wu H. Revisiting Attention for Multivariate Time Series Forecasting. Proceedings of the AAAI Conference on Artificial Intelligence. 2025;39(20):21528-21535. doi:10.1609/aaai.v39i20.35455

BibTeX
@article{Wu2025,
  author = {Wu, Haixiang},
  title = {Revisiting Attention for Multivariate Time Series Forecasting},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2025},
  volume = {39},
  number = {20},
  pages = {21528--21535},
  doi = {10.1609/aaai.v39i20.35455}
}
Kostenlos starten →