RS-LLaVA: A Large Vision-Language Model for Joint Captioning and Question Answering in Remote Sensing Imagery
Remote Sensing · 23.04.2024 · 16 (9) · 1477
DOI: 10.3390/rs16091477 ↗Kurz gefasst
Fachartikel in der Zeitschrift Remote Sensing, online erschienen am 23.04.2024. Verfasst von Yakoub Bazi und 4 weiteren Autor:innen. Die Arbeit wurde bisher 135-mal zitiert (Crossref) und nennt 67 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: King Saud University.
Abstract (Originalsprache)
In this paper, we delve into the innovative application of large language models (LLMs) and their extension, large vision-language models (LVLMs), in the field of remote sensing (RS) image analysis. We particularly emphasize their multi-tasking potential with a focus on image captioning and visual question answering (VQA). In particular, we introduce an improved version of the Large Language and Vision Assistant Model (LLaVA), specifically adapted for RS imagery through a low-rank adaptation approach. To evaluate the model performance, we create the RS-instructions dataset, a comprehensive benchmark dataset that integrates four diverse single-task datasets related to captioning and VQA. The experimental results confirm the model’s effectiveness, marking a step forward toward the development of efficient multi-task models for RS image analysis.
Förderung
- King Saud University
Zitieren
APA
Bazi, Y., Bashmal, L., Rahhal, M. M. A., Ricci, R., & Melgani, F. (2024). RS-LLaVA: A Large Vision-Language Model for Joint Captioning and Question Answering in Remote Sensing Imagery. Remote Sensing, 16(9), 1477. https://doi.org/10.3390/rs16091477
Vancouver
Bazi Y, Bashmal L, Rahhal MMA, Ricci R, Melgani F. RS-LLaVA: A Large Vision-Language Model for Joint Captioning and Question Answering in Remote Sensing Imagery. Remote Sensing. 2024;16(9):1477. doi:10.3390/rs16091477
BibTeX
@article{Bazi2024,
author = {Bazi, Yakoub and Bashmal, Laila and Rahhal, Mohamad Mahmoud Al and Ricci, Riccardo and Melgani, Farid},
title = {RS-LLaVA: A Large Vision-Language Model for Joint Captioning and Question Answering in Remote Sensing Imagery},
journal = {Remote Sensing},
year = {2024},
volume = {16},
number = {9},
pages = {1477},
doi = {10.3390/rs16091477}
}
EUVIMED