EUVIMEDOpdag EUVIMED
Open access · CC BY-NC-ND 4.0

Evaluating Transformer-Based Foundation Models for Time-Series Forecasting Across Multiple Horizons

Miranda Harizaj, Alfons Harizaj, Olgerta Idrizi

TEM Journal · 2026-05-27 · 1226

DOI: 10.18421/tem152-24 ↗

At a glance

Journal article in TEM Journal, published online on 2026-05-27. Written by Miranda Harizaj and 2 co-authors. The full text is freely available (open access, CC BY-NC-ND 4.0).

AbstractBibliografiske oplysningerCiteRelated studies

Abstract (original language)

Recent studies show that transformerbased architectures and foundation models have achieved promising results in time-series forecasting, yet their advantages over traditional machine-learning methods remain inconsistent across domains and forecasting horizons. This study investigates whether modern transformer-based models provide systematic performance gains compared to classical regression approaches and examines four representative models as SAMFormer, TimesFM, Time-MoE, and TimeGPT and benchmark them against established baselines including Linear Regression, MLPRegressor and ensemble methods. In the paper it is hypothesized that transformer-based models outperform traditional methods, particularly for longer forecasting horizons and datasets with strong temporal dependencies. An experimental evaluation is conducted across four realworld datasets from weather, finance, energy, and healthcare domains, using multiple context and prediction length settings. Model performance is assessed using standard error-based and distributionbased metrics. The results show that transformer-based models generally outperform regression baselines, with SAMFormer demonstrating the most stable and consistent performance across datasets and horizons. TimeGPT excels in short-term forecasting, while TimesFM exhibits limited robustness, especially for longer horizons. Fine-tuning yields mixed benefits, depending on the dataset and model architecture. Overall, the findings provide an evidence-based assessment of when transformer-based forecasting is advantageous and when simpler models remain competitive, offering practical guidance for model selection in real-world time-series applications.

Cite

APA

Harizaj, M., Harizaj, A., & Idrizi, O. (2026). Evaluating Transformer-Based Foundation Models for Time-Series Forecasting Across Multiple Horizons. TEM Journal, 1226. https://doi.org/10.18421/tem152-24

Vancouver

Harizaj M, Harizaj A, Idrizi O. Evaluating Transformer-Based Foundation Models for Time-Series Forecasting Across Multiple Horizons. TEM Journal. 2026:1226. doi:10.18421/tem152-24

BibTeX
@article{Harizaj2026,
  author = {Harizaj, Miranda and Harizaj, Alfons and Idrizi, Olgerta},
  title = {Evaluating Transformer-Based Foundation Models for Time-Series Forecasting Across Multiple Horizons},
  journal = {TEM Journal},
  year = {2026},
  pages = {1226},
  doi = {10.18421/tem152-24}
}
Start for free →