ICL Characterization of Climate Foundation Models: When Can Transformers Learn Weather and Climate?
AIR Journal of Mathematics and Computational Sciences · 04.03.2026 · 2026 (AIRMCS2026405)
DOI: 10.65737/airmcs2026405 ↗Kurz gefasst
Fachartikel in der Zeitschrift AIR Journal of Mathematics and Computational Sciences, online erschienen am 04.03.2026. Verfasst von Mosab Hawarey. Die Arbeit wurde bisher 7-mal zitiert (Crossref) und nennt 30 Literaturangaben.
Abstract (Originalsprache)
Climate foundation models (FMs) have achieved remarkable success in weather forecasting, yet exhibit puzzling performance gaps between tasks: deterministic field prediction rivals operational numerical weather prediction, while extreme event detection lags significantly behind. We provide a theoretical explanation through the lens of in-context learning (ICL). Extending the ICL characterization framework to spatiotemporal climate data, we prove the Climate Prediction Dichotomy Theorem: every natural climate task falls into exactly one of two complexity categories. Type A (ICL-Easy) tasks—including temperature, pressure, and wind field prediction—admit additive sufficient statistics enabling attention-based computation with sample complexity nICL = Θ(nERM). Type C (ICL-Hard) tasks—including extreme event detection, tipping point identification, and compound event localization—require combinatorial sufficient statistics that provably exceed the computational capacity of constant-depth polynomial-size transformers when the number of simultaneous events J exceeds a threshold J* = O(log log n) ≈ 3–5. We establish the predictability horizon constraint: weather forecasting is ICL-Easy for lead times τ < τL ≈ 14 days, while climate statistics remain ICL-accessible but individual trajectories are fundamentally unpredictable. Our analysis yields six testable predictions about climate FM behavior and five deployment guidelines distinguishing when ICL suffices versus when fine-tuning is required. The dichotomy provides a principled foundation for understanding why Pangu-Weather and GraphCast excel at field prediction while struggling with event detection—and guides the design of next-generation climate AI systems.
Zitieren
APA
Hawarey, M. (2026). ICL Characterization of Climate Foundation Models: When Can Transformers Learn Weather and Climate?. AIR Journal of Mathematics and Computational Sciences, 2026(AIRMCS2026405). https://doi.org/10.65737/airmcs2026405
Vancouver
Hawarey M. ICL Characterization of Climate Foundation Models: When Can Transformers Learn Weather and Climate?. AIR Journal of Mathematics and Computational Sciences. 2026;2026(AIRMCS2026405). doi:10.65737/airmcs2026405
BibTeX
@article{Hawarey2026,
author = {Hawarey, Mosab},
title = {ICL Characterization of Climate Foundation Models: When Can Transformers Learn Weather and Climate?},
journal = {AIR Journal of Mathematics and Computational Sciences},
year = {2026},
volume = {2026},
number = {AIRMCS2026405},
doi = {10.65737/airmcs2026405}
}
EUVIMED