Fourier-mixed window attention for efficient and robust long sequence time-series forecasting
Frontiers in Applied Mathematics and Statistics · 19.05.2025 · 11
DOI: 10.3389/fams.2025.1600136 ↗Kurz gefasst
Fachartikel in der Zeitschrift Frontiers in Applied Mathematics and Statistics, online erschienen am 19.05.2025. Verfasst von Nhat Thanh Tran und 1 weiteren Autor:innen. Die Arbeit wurde bisher 4-mal zitiert (Crossref) und nennt 27 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Science Foundation, Qualcomm.
Abstract (Originalsprache)
We study a fast local-global window-based attention method to accelerate Informer for long sequence time-series forecasting (LSTF) in a robust manner. While window attention being local is a considerable computational saving, it lacks the ability to capture global token information which is compensated by a subsequent Fourier transform block. Our method, named FWin, does not rely on query sparsity hypothesis and an empirical approximation underlying the ProbSparse attention of Informer. Experiments on univariate and multivariate datasets show that FWin transformers improve the overall prediction accuracies of Informer while accelerating its inference speeds by 1.6 to 2 times. On strongly non-stationary data (power grid and dengue disease data), FWin outperforms Informer and recent SOTAs thereby demonstrating its superior robustness . We give mathematical definition of FWin attention, and prove its equivalency to the canonical full attention under the block diagonal invertibility (BDI) condition of the attention matrix. The BDI is verified to hold with high probability on benchmark datasets experimentally.
Förderung
- National Science Foundation
- Qualcomm
Zitieren
APA
Tran, N. T., & Xin, J. (2025). Fourier-mixed window attention for efficient and robust long sequence time-series forecasting. Frontiers in Applied Mathematics and Statistics, 11. https://doi.org/10.3389/fams.2025.1600136
Vancouver
Tran NT, Xin J. Fourier-mixed window attention for efficient and robust long sequence time-series forecasting. Frontiers in Applied Mathematics and Statistics. 2025;11. doi:10.3389/fams.2025.1600136
BibTeX
@article{Tran2025,
author = {Tran, Nhat Thanh and Xin, Jack},
title = {Fourier-mixed window attention for efficient and robust long sequence time-series forecasting},
journal = {Frontiers in Applied Mathematics and Statistics},
year = {2025},
volume = {11},
doi = {10.3389/fams.2025.1600136}
}
EUVIMED