EUVIMEDEUVIMED entdecken
Crossref

Towards Knowledge Guided Pretraining Approaches for Multimodal Foundation Models: Applications in Remote Sensing

Praveen Ravirathinam, Ajitesh Parthasarathy, Ankush Khandelwal, Rahul Ghosh, Vipin Kumar

ACM Transactions on Spatial Algorithms and Systems · 31.03.2026

DOI: 10.1145/3806229 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM Transactions on Spatial Algorithms and Systems, online erschienen am 31.03.2026. Verfasst von Praveen Ravirathinam und 4 weiteren Autor:innen. Die Arbeit nennt 62 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Self-supervised learning has emerged as a powerful paradigm for pretraining foundation models using large-scale data. Existing pretraining approaches predominantly rely on masked reconstruction or next-token prediction strategies, demonstrating strong performance across various downstream tasks, including geoscience applications. However, these approaches do not fully capture the knowledge of causal interplay between different geospatial and environmental variables. To address this limitation, we propose Knowledge Guided Variable-Step Forecasting (KG-VSF), a novel pretraining task that models forecasting as a conditional generation task, where driver variables (e.g., weather) inform the prediction of response variables (e.g., satellite imagery). We demonstrate that pretraining in such a fashion leads to strong embeddings which give enhanced performance when finetuned on downstream tasks where capturing this causality matters such as pixel wise crop type mapping, soil moisture estimation and forecasting, missing image prediction, and future image forecasting when compared to finetuning embeddings from other standard pretraining approaches.

Zitieren

APA

Ravirathinam, P., Parthasarathy, A., Khandelwal, A., Ghosh, R., & Kumar, V. (2026). Towards Knowledge Guided Pretraining Approaches for Multimodal Foundation Models: Applications in Remote Sensing. ACM Transactions on Spatial Algorithms and Systems. https://doi.org/10.1145/3806229

Vancouver

Ravirathinam P, Parthasarathy A, Khandelwal A, Ghosh R, Kumar V. Towards Knowledge Guided Pretraining Approaches for Multimodal Foundation Models: Applications in Remote Sensing. ACM Transactions on Spatial Algorithms and Systems. 2026. doi:10.1145/3806229

BibTeX
@article{Ravirathinam2026,
  author = {Ravirathinam, Praveen and Parthasarathy, Ajitesh and Khandelwal, Ankush and Ghosh, Rahul and Kumar, Vipin},
  title = {Towards Knowledge Guided Pretraining Approaches for Multimodal Foundation Models: Applications in Remote Sensing},
  journal = {ACM Transactions on Spatial Algorithms and Systems},
  year = {2026},
  doi = {10.1145/3806229}
}
Kostenlos starten →