EUVIMEDEUVIMED entdecken
Crossref

ICL Characterization of Climate Foundation Models: When Can Transformers Learn Weather and Climate?

Mosab Hawarey

AIR Journal of Mathematics and Computational Sciences · 04.03.2026 · 2026 (AIRMCS2026405)

DOI: 10.65737/airmcs2026405 ↗

Kurz gefasst

Fachartikel in der Zeitschrift AIR Journal of Mathematics and Computational Sciences, online erschienen am 04.03.2026. Verfasst von Mosab Hawarey. Die Arbeit wurde bisher 7-mal zitiert (Crossref) und nennt 30 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Climate foundation models (FMs) have achieved remarkable success in weather forecasting, yet exhibit puzzling performance gaps between tasks: deterministic field prediction rivals operational numerical weather prediction, while extreme event detection lags significantly behind. We provide a theoretical explanation through the lens of in-context learning (ICL). Extending the ICL characterization framework to spatiotemporal climate data, we prove the Climate Prediction Dichotomy Theorem: every natural climate task falls into exactly one of two complexity categories. Type A (ICL-Easy) tasks—including temperature, pressure, and wind field prediction—admit additive sufficient statistics enabling attention-based computation with sample complexity nICL = Θ(nERM). Type C (ICL-Hard) tasks—including extreme event detection, tipping point identification, and compound event localization—require combinatorial sufficient statistics that provably exceed the computational capacity of constant-depth polynomial-size transformers when the number of simultaneous events J exceeds a threshold J* = O(log log n) ≈ 3–5. We establish the predictability horizon constraint: weather forecasting is ICL-Easy for lead times τ < τL ≈ 14 days, while climate statistics remain ICL-accessible but individual trajectories are fundamentally unpredictable. Our analysis yields six testable predictions about climate FM behavior and five deployment guidelines distinguishing when ICL suffices versus when fine-tuning is required. The dichotomy provides a principled foundation for understanding why Pangu-Weather and GraphCast excel at field prediction while struggling with event detection—and guides the design of next-generation climate AI systems.

Zitieren

APA

Hawarey, M. (2026). ICL Characterization of Climate Foundation Models: When Can Transformers Learn Weather and Climate?. AIR Journal of Mathematics and Computational Sciences, 2026(AIRMCS2026405). https://doi.org/10.65737/airmcs2026405

Vancouver

Hawarey M. ICL Characterization of Climate Foundation Models: When Can Transformers Learn Weather and Climate?. AIR Journal of Mathematics and Computational Sciences. 2026;2026(AIRMCS2026405). doi:10.65737/airmcs2026405

BibTeX
@article{Hawarey2026,
  author = {Hawarey, Mosab},
  title = {ICL Characterization of Climate Foundation Models: When Can Transformers Learn Weather and Climate?},
  journal = {AIR Journal of Mathematics and Computational Sciences},
  year = {2026},
  volume = {2026},
  number = {AIRMCS2026405},
  doi = {10.65737/airmcs2026405}
}
Kostenlos starten →