EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

A Practical Probabilistic Benchmark for AI Weather Models

Noah D. Brenowitz, Yair Cohen, Jaideep Pathak, Ankur Mahesh, Boris Bonev, Thorsten Kurth, Dale R. Durran, Peter Harrington, Michael S. Pritchard

Geophysical Research Letters · 16.04.2025 · 52 (7)

DOI: 10.1029/2024gl113656 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Geophysical Research Letters, online erschienen am 08.04.2025. Verfasst von Noah D. Brenowitz und 8 weiteren Autor:innen. Die Arbeit wurde bisher 27-mal zitiert (Crossref) und nennt 41 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Advanced Scientific Computing Research, Office of Science, National Energy Research Scientific Computing Center.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Since the weather is chaotic, it is necessary to forecast an ensemble of future states. Recently, multiple AI weather models have emerged claiming breakthroughs in deterministic skill. Unfortunately, it is hard to fairly compare ensembles of AI forecasts because variations in ensembling methodology become confounding and the baseline data volume is immense. We address this by scoring lagged initial condition ensembles—whereby an ensemble can be constructed from a library of deterministic hindcasts. This allows the first parameter‐free intercomparison of leading AI weather models' probabilistic skill against an operational baseline. Lagged ensembles of the two leading AI weather models, GraphCast and Pangu, perform similarly even though the former outperforms the latter in deterministic scoring. These results are elaborated upon by sensitivity tests showing that commonly used multiple time‐step loss functions damage ensemble calibration.

Förderung

  • Advanced Scientific Computing Research
  • Office of Science
  • National Energy Research Scientific Computing Center

Zitieren

APA

Brenowitz, N. D., Cohen, Y., Pathak, J., Mahesh, A., Bonev, B., Kurth, T., Durran, D. R., Harrington, P., & Pritchard, M. S. (2025). A Practical Probabilistic Benchmark for AI Weather Models. Geophysical Research Letters, 52(7). https://doi.org/10.1029/2024gl113656

Vancouver

Brenowitz ND, Cohen Y, Pathak J, Mahesh A, Bonev B, Kurth T, et al. A Practical Probabilistic Benchmark for AI Weather Models. Geophysical Research Letters. 2025;52(7). doi:10.1029/2024gl113656

BibTeX
@article{Brenowitz2025,
  author = {Brenowitz, Noah D. and Cohen, Yair and Pathak, Jaideep and Mahesh, Ankur and Bonev, Boris and Kurth, Thorsten and Durran, Dale R. and Harrington, Peter and Pritchard, Michael S.},
  title = {A Practical Probabilistic Benchmark for AI Weather Models},
  journal = {Geophysical Research Letters},
  year = {2025},
  volume = {52},
  number = {7},
  doi = {10.1029/2024gl113656}
}
Kostenlos starten →