TFB: Towards Comprehensive and Fair Benchmarking of Time Series Forecasting Methods
Proceedings of the VLDB Endowment · 2024-05 · 17 (9) · 2363-2377
DOI: 10.14778/3665844.3665863 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the VLDB Endowment, online erschienen am 06.08.2024. Verfasst von Xiangfei Qiu und 10 weiteren Autor:innen. Die Arbeit wurde bisher 193-mal zitiert (Crossref) und nennt 106 Literaturangaben.
Abstract (Originalsprache)
Time series are generated in diverse domains such as economic, traffic, health, and energy, where forecasting of future values has numerous important applications. Not surprisingly, many forecasting methods are being proposed. To ensure progress, it is essential to be able to study and compare such methods empirically in a comprehensive and reliable manner. To achieve this, we propose TFB, an automated benchmark for Time Series Forecasting (TSF) methods. TFB advances the state-of-the-art by addressing shortcomings related to datasets, comparison methods, and evaluation pipelines: 1) insufficient coverage of data domains, 2) stereotype bias against traditional methods, and 3) inconsistent and inflexible pipelines. To achieve better domain coverage, we include datasets from 10 different domains : traffic, electricity, energy, the environment, nature, economic, stock markets, banking, health, and the web. We also provide a time series characterization to ensure that the selected datasets are comprehensive. To remove biases against some methods, we include a diverse range of methods, including statistical learning, machine learning, and deep learning methods, and we also support a variety of evaluation strategies and metrics to ensure a more comprehensive evaluations of different methods. To support the integration of different methods into the benchmark and enable fair comparisons, TFB features a flexible and scalable pipeline that eliminates biases. Next, we employ TFB to perform a thorough evaluation of 21 Univariate Time Series Forecasting (UTSF) methods on 8,068 univariate time series and 14 Multivariate Time Series Forecasting (MTSF) methods on 25 datasets. The results offer a deeper understanding of the forecasting methods, allowing us to better select the ones that are most suitable for particular datasets and settings. Overall, TFB and this evaluation provide researchers with improved means of designing new TSF methods.
Zitieren
APA
Qiu, X., Hu, J., Zhou, L., Wu, X., Du, J., Zhang, B., Guo, C., Zhou, A., Jensen, C. S., Sheng, Z., & Yang, B. (2024). TFB: Towards Comprehensive and Fair Benchmarking of Time Series Forecasting Methods. Proceedings of the VLDB Endowment, 17(9), 2363-2377. https://doi.org/10.14778/3665844.3665863
Vancouver
Qiu X, Hu J, Zhou L, Wu X, Du J, Zhang B, et al. TFB: Towards Comprehensive and Fair Benchmarking of Time Series Forecasting Methods. Proceedings of the VLDB Endowment. 2024;17(9):2363-2377. doi:10.14778/3665844.3665863
BibTeX
@article{Qiu2024,
author = {Qiu, Xiangfei and Hu, Jilin and Zhou, Lekui and Wu, Xingjian and Du, Junyang and Zhang, Buang and Guo, Chenjuan and Zhou, Aoying and Jensen, Christian S. and Sheng, Zhenli and Yang, Bin},
title = {TFB: Towards Comprehensive and Fair Benchmarking of Time Series Forecasting Methods},
journal = {Proceedings of the VLDB Endowment},
year = {2024},
volume = {17},
number = {9},
pages = {2363--2377},
doi = {10.14778/3665844.3665863}
}
EUVIMED