A Practical Probabilistic Benchmark for AI Weather Models
Geophysical Research Letters · 16.04.2025 · 52 (7)
DOI: 10.1029/2024gl113656 ↗Kurz gefasst
Fachartikel in der Zeitschrift Geophysical Research Letters, online erschienen am 08.04.2025. Verfasst von Noah D. Brenowitz und 8 weiteren Autor:innen. Die Arbeit wurde bisher 27-mal zitiert (Crossref) und nennt 41 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Advanced Scientific Computing Research, Office of Science, National Energy Research Scientific Computing Center.
Abstract (Originalsprache)
Since the weather is chaotic, it is necessary to forecast an ensemble of future states. Recently, multiple AI weather models have emerged claiming breakthroughs in deterministic skill. Unfortunately, it is hard to fairly compare ensembles of AI forecasts because variations in ensembling methodology become confounding and the baseline data volume is immense. We address this by scoring lagged initial condition ensembles—whereby an ensemble can be constructed from a library of deterministic hindcasts. This allows the first parameter‐free intercomparison of leading AI weather models' probabilistic skill against an operational baseline. Lagged ensembles of the two leading AI weather models, GraphCast and Pangu, perform similarly even though the former outperforms the latter in deterministic scoring. These results are elaborated upon by sensitivity tests showing that commonly used multiple time‐step loss functions damage ensemble calibration.
Förderung
- Advanced Scientific Computing Research
- Office of Science
- National Energy Research Scientific Computing Center
Zitieren
APA
Brenowitz, N. D., Cohen, Y., Pathak, J., Mahesh, A., Bonev, B., Kurth, T., Durran, D. R., Harrington, P., & Pritchard, M. S. (2025). A Practical Probabilistic Benchmark for AI Weather Models. Geophysical Research Letters, 52(7). https://doi.org/10.1029/2024gl113656
Vancouver
Brenowitz ND, Cohen Y, Pathak J, Mahesh A, Bonev B, Kurth T, et al. A Practical Probabilistic Benchmark for AI Weather Models. Geophysical Research Letters. 2025;52(7). doi:10.1029/2024gl113656
BibTeX
@article{Brenowitz2025,
author = {Brenowitz, Noah D. and Cohen, Yair and Pathak, Jaideep and Mahesh, Ankur and Bonev, Boris and Kurth, Thorsten and Durran, Dale R. and Harrington, Peter and Pritchard, Michael S.},
title = {A Practical Probabilistic Benchmark for AI Weather Models},
journal = {Geophysical Research Letters},
year = {2025},
volume = {52},
number = {7},
doi = {10.1029/2024gl113656}
}
EUVIMED