Revisiting Attention for Multivariate Time Series Forecasting
Proceedings of the AAAI Conference on Artificial Intelligence · 11.04.2025 · 39 (20) · 21528-21535
DOI: 10.1609/aaai.v39i20.35455 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 11.04.2025. Verfasst von Haixiang Wu. Die Arbeit wurde bisher 15-mal zitiert (Crossref).
Abstract (Originalsprache)
Current Transformer methods for Multivariate Time-Series Forecasting (MTSF) are all based on the conventional attention mechanism. They involve sequence embedding and performing a linear projection for Q, K, and V, and then computing attention within this latent space. We have not yet delved into the attention mechanism to explore whether such a mapping space is optimal for MTSF. To investigate this issue, we first propose Frequency Spectrum attention (FSatten), a novel attention mechanism based on the frequency domain space. It employs the Fourier transform for embedding and introduces Multi-head Spectrum Scaling (MSS) to replace the conventional linear mapping for Q and K. FSatten can accurately capture the periodic dependencies between sequences and outperform the conventional attention, without necessitating changes to mainstream architectures. We further design a more general method dubbed Scaled Orthogonal attention (SOatten). We propose an orthogonal embedding and a Head-Coupling Convolution (HCC) based on the neighboring similarity bias to guide the model in learning comprehensive dependency patterns. Experiments show that FSatten and SOatten surpass the SOTA which uses conventional attention, making it a good alternative as a basic attention mechanism for MTSF.
Zitieren
APA
Wu, H. (2025). Revisiting Attention for Multivariate Time Series Forecasting. Proceedings of the AAAI Conference on Artificial Intelligence, 39(20), 21528-21535. https://doi.org/10.1609/aaai.v39i20.35455
Vancouver
Wu H. Revisiting Attention for Multivariate Time Series Forecasting. Proceedings of the AAAI Conference on Artificial Intelligence. 2025;39(20):21528-21535. doi:10.1609/aaai.v39i20.35455
BibTeX
@article{Wu2025,
author = {Wu, Haixiang},
title = {Revisiting Attention for Multivariate Time Series Forecasting},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2025},
volume = {39},
number = {20},
pages = {21528--21535},
doi = {10.1609/aaai.v39i20.35455}
}
EUVIMED