Towards Knowledge Guided Pretraining Approaches for Multimodal Foundation Models: Applications in Remote Sensing
ACM Transactions on Spatial Algorithms and Systems · 31.03.2026
DOI: 10.1145/3806229 ↗Kurz gefasst
Fachartikel in der Zeitschrift ACM Transactions on Spatial Algorithms and Systems, online erschienen am 31.03.2026. Verfasst von Praveen Ravirathinam und 4 weiteren Autor:innen. Die Arbeit nennt 62 Literaturangaben.
Abstract (Originalsprache)
Self-supervised learning has emerged as a powerful paradigm for pretraining foundation models using large-scale data. Existing pretraining approaches predominantly rely on masked reconstruction or next-token prediction strategies, demonstrating strong performance across various downstream tasks, including geoscience applications. However, these approaches do not fully capture the knowledge of causal interplay between different geospatial and environmental variables. To address this limitation, we propose Knowledge Guided Variable-Step Forecasting (KG-VSF), a novel pretraining task that models forecasting as a conditional generation task, where driver variables (e.g., weather) inform the prediction of response variables (e.g., satellite imagery). We demonstrate that pretraining in such a fashion leads to strong embeddings which give enhanced performance when finetuned on downstream tasks where capturing this causality matters such as pixel wise crop type mapping, soil moisture estimation and forecasting, missing image prediction, and future image forecasting when compared to finetuning embeddings from other standard pretraining approaches.
Zitieren
APA
Ravirathinam, P., Parthasarathy, A., Khandelwal, A., Ghosh, R., & Kumar, V. (2026). Towards Knowledge Guided Pretraining Approaches for Multimodal Foundation Models: Applications in Remote Sensing. ACM Transactions on Spatial Algorithms and Systems. https://doi.org/10.1145/3806229
Vancouver
Ravirathinam P, Parthasarathy A, Khandelwal A, Ghosh R, Kumar V. Towards Knowledge Guided Pretraining Approaches for Multimodal Foundation Models: Applications in Remote Sensing. ACM Transactions on Spatial Algorithms and Systems. 2026. doi:10.1145/3806229
BibTeX
@article{Ravirathinam2026,
author = {Ravirathinam, Praveen and Parthasarathy, Ajitesh and Khandelwal, Ankush and Ghosh, Rahul and Kumar, Vipin},
title = {Towards Knowledge Guided Pretraining Approaches for Multimodal Foundation Models: Applications in Remote Sensing},
journal = {ACM Transactions on Spatial Algorithms and Systems},
year = {2026},
doi = {10.1145/3806229}
}
EUVIMED