On the Opportunities and Challenges of Foundation Models for GeoAI (Vision Paper)
ACM Transactions on Spatial Algorithms and Systems · 30.06.2024 · 10 (2) · 1-46
DOI: 10.1145/3653070 ↗Kurz gefasst
Fachartikel in der Zeitschrift ACM Transactions on Spatial Algorithms and Systems, 2024-07. Verfasst von Gengchen Mai und 13 weiteren Autor:innen. Die Arbeit wurde bisher 143-mal zitiert (Crossref) und nennt 225 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Knut and Alice Wallenberg Foundation, National Science Foundation–funded AI Institute, Intelligent Cyberinfrastructure with Computational Learning in the Environment.
Abstract (Originalsprache)
Large pre-trained models, also known as foundation models (FMs), are trained in a task-agnostic manner on large-scale data and can be adapted to a wide range of downstream tasks by fine-tuning, few-shot, or even zero-shot learning. Despite their successes in language and vision tasks, we have not yet seen an attempt to develop foundation models for geospatial artificial intelligence (GeoAI). In this work, we explore the promises and challenges of developing multimodal foundation models for GeoAI. We first investigate the potential of many existing FMs by testing their performances on seven tasks across multiple geospatial domains, including Geospatial Semantics, Health Geography, Urban Geography, and Remote Sensing. Our results indicate that on several geospatial tasks that only involve text modality, such as toponym recognition, location description recognition, and US state-level/county-level dementia time series forecasting, the task-agnostic large learning models (LLMs) can outperform task-specific fully supervised models in a zero-shot or few-shot learning setting. However, on other geospatial tasks, especially tasks that involve multiple data modalities (e.g., POI-based urban function classification, street view image–based urban noise intensity classification, and remote sensing image scene classification), existing FMs still underperform task-specific models. Based on these observations, we propose that one of the major challenges of developing an FM for GeoAI is to address the multimodal nature of geospatial tasks. After discussing the distinct challenges of each geospatial data modality, we suggest the possibility of a multimodal FM that can reason over various types of geospatial data through geospatial alignments. We conclude this article by discussing the unique risks and challenges to developing such a model for GeoAI.
Förderung
- Knut and Alice Wallenberg Foundation
- National Science Foundation–funded AI Institute
- Intelligent Cyberinfrastructure with Computational Learning in the Environment
Zitieren
APA
Mai, G., Huang, W., Sun, J., Song, S., Mishra, D., Liu, N., Gao, S., Liu, T., Cong, G., Hu, Y., Cundy, C., Li, Z., Zhu, R., & Lao, N. (2024). On the Opportunities and Challenges of Foundation Models for GeoAI (Vision Paper). ACM Transactions on Spatial Algorithms and Systems, 10(2), 1-46. https://doi.org/10.1145/3653070
Vancouver
Mai G, Huang W, Sun J, Song S, Mishra D, Liu N, et al. On the Opportunities and Challenges of Foundation Models for GeoAI (Vision Paper). ACM Transactions on Spatial Algorithms and Systems. 2024;10(2):1-46. doi:10.1145/3653070
BibTeX
@article{Mai2024,
author = {Mai, Gengchen and Huang, Weiming and Sun, Jin and Song, Suhang and Mishra, Deepak and Liu, Ninghao and Gao, Song and Liu, Tianming and Cong, Gao and Hu, Yingjie and Cundy, Chris and Li, Ziyuan and Zhu, Rui and Lao, Ni},
title = {On the Opportunities and Challenges of Foundation Models for GeoAI (Vision Paper)},
journal = {ACM Transactions on Spatial Algorithms and Systems},
year = {2024},
volume = {10},
number = {2},
pages = {1--46},
doi = {10.1145/3653070}
}
EUVIMED