EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC-ND 4.0

Using cross‐validation methods to select time series models: Promises and pitfalls

Siwei Liu iD, Di Jody Zhou

British Journal of Mathematical and Statistical Psychology · 2024-05 · 77 (2) · 337-355

DOI: 10.1111/bmsp.12330 ↗

Kurz gefasst

Fachartikel in der Zeitschrift British Journal of Mathematical and Statistical Psychology, online erschienen am 07.12.2023. Verfasst von Siwei Liu und 1 weiteren Autor:innen. Beteiligte Einrichtung: Human Development and Family Studies, Department of Human Ecology, University of California at Davis, Davis, California, USA. Hauptthemen laut MEDLINE: Models, Statistical, Research Design. Die Arbeit wurde bisher 31-mal zitiert (Crossref) und nennt 47 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0). Gefördert von: USDA-NIFA Hatch Grant CA-D-HCE-2195-H.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Vector autoregressive (VAR) modelling is widely employed in psychology for time series analyses of dynamic processes. However, the typically short time series in psychological studies can lead to overfitting of VAR models, impairing their predictive ability on unseen samples. Cross‐validation (CV) methods are commonly recommended for assessing the predictive ability of statistical models. However, it is unclear how the performance of CV is affected by characteristics of time series data and the fitted models. In this simulation study, we examine the ability of two CV methods, namely,10‐fold CV and blocked CV, in estimating the prediction errors of three time series models with increasing complexity (person‐mean, AR, and VAR), and evaluate how their performance is affected by data characteristics. We then compare these CV methods to the traditional methods using the Akaike (AIC) and Bayesian (BIC) information criteria in their accuracy of selecting the most predictive models. We find that CV methods tend to underestimate prediction errors of simpler models, but overestimate prediction errors of VAR models, particularly when the number of observations is small. Nonetheless, CV methods, especially blocked CV, generally outperform the AIC and BIC. We conclude our study with a discussion on the implications of the findings and provide helpful guidelines for practice.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Models, StatisticalResearch DesignHumansTime FactorsBayes TheoremComputer Simulation

Stichwörter der Autor:innen

autoregressive modelcross‐validationinformation criteriatime seriesvector autoregressive model

Autor:innen und Institutionen

  • Human Development and Family Studies, Department of Human Ecology, University of California at Davis, Davis, California, USA.

ORCID-Profile: Siwei Liu

Förderung

  • USDA-NIFA Hatch Grant CA-D-HCE-2195-H

Zitieren

APA

Liu, S., & Zhou, D. J. (2024). Using cross‐validation methods to select time series models: Promises and pitfalls. British Journal of Mathematical and Statistical Psychology, 77(2), 337-355. https://doi.org/10.1111/bmsp.12330

Vancouver

Liu S, Zhou DJ. Using cross‐validation methods to select time series models: Promises and pitfalls. British Journal of Mathematical and Statistical Psychology. 2024;77(2):337-355. doi:10.1111/bmsp.12330

BibTeX
@article{Liu2024,
  author = {Liu, Siwei and Zhou, Di Jody},
  title = {Using cross‐validation methods to select time series models: Promises and pitfalls},
  journal = {British Journal of Mathematical and Statistical Psychology},
  year = {2024},
  volume = {77},
  number = {2},
  pages = {337--355},
  doi = {10.1111/bmsp.12330}
}
Kostenlos starten →