Co-LLaVA: Efficient Remote Sensing Visual Question Answering via Model Collaboration
Remote Sensing · 29.01.2025 · 17 (3) · 466
DOI: 10.3390/rs17030466 ↗Kurz gefasst
Fachartikel in der Zeitschrift Remote Sensing, online erschienen am 29.01.2025. Verfasst von Fan Liu und 5 weiteren Autor:innen. Die Arbeit wurde bisher 13-mal zitiert (Crossref) und nennt 61 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Fundamental Research Funds for the Central Universities, National Nature Science Foundation of China, Aeronautical Science Fund.
Abstract (Originalsprache)
Large vision language models (LVLMs) are built upon large language models (LLMs) and incorporate non-textual modalities; they can perform various multimodal tasks. Applying LVLMs in remote sensing (RS) visual question answering (VQA) tasks can take advantage of the powerful capabilities to promote the development of VQA in RS. However, due to the greater complexity of remote sensing images compared to natural images, general-domain LVLMs tend to perform poorly in RS scenarios and are prone to hallucination phenomena. Multi-agent debate for collaborative reasoning is commonly utilized to mitigate hallucination phenomena. Although this method is effective, it comes with a significant computational burden (e.g., high CPU/GPU demands and slow inference speed). To address these limitations, we propose Co-LLaVA, a model specifically designed for RS VQA tasks. Specifically, Co-LLaVA employs model collaboration between Large Language and Vision Assistant (LLaVA-v1.5) and Contrastive Captioners (CoCas). It combines LVLM with a lightweight generative model, reducing computational burden compared to multi-agent debate. Additionally, through high-dimensional multi-scale features and higher-resolution images, Co-LLaVA can enhance the perception of details in RS images. Experimental results demonstrate the significant performance improvements of our Co-LLaVA over existing LVLMs (e.g., Geochat, RSGPT) on multiple metrics of four RS VQA datasets (e.g., +3% over SkySenseGPT on “Rural/Urban” accuracy in the test set of RSVQA-LR dataset).
Förderung
- Fundamental Research Funds for the Central Universities
- National Nature Science Foundation of China
- Aeronautical Science Fund
- Joint Fund of Ministry of Education for Equipment Pre-research
- Water Science and Technology Project of Jiangsu Province
- Qinglan Project of Jiangsu Province, Changzhou science and technology project
- Postgraduate Research & Practice Innovation Program of Jiangsu Province
Zitieren
APA
Liu, F., Dai, W., Zhang, C., Zhu, J., Yao, L., & Li, X. (2025). Co-LLaVA: Efficient Remote Sensing Visual Question Answering via Model Collaboration. Remote Sensing, 17(3), 466. https://doi.org/10.3390/rs17030466
Vancouver
Liu F, Dai W, Zhang C, Zhu J, Yao L, Li X. Co-LLaVA: Efficient Remote Sensing Visual Question Answering via Model Collaboration. Remote Sensing. 2025;17(3):466. doi:10.3390/rs17030466
BibTeX
@article{Liu2025,
author = {Liu, Fan and Dai, Wenwen and Zhang, Chuanyi and Zhu, Jiale and Yao, Liang and Li, Xin},
title = {Co-LLaVA: Efficient Remote Sensing Visual Question Answering via Model Collaboration},
journal = {Remote Sensing},
year = {2025},
volume = {17},
number = {3},
pages = {466},
doi = {10.3390/rs17030466}
}
EUVIMED