EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

RS-LLaVA: A Large Vision-Language Model for Joint Captioning and Question Answering in Remote Sensing Imagery

Yakoub Bazi, Laila Bashmal, Mohamad Mahmoud Al Rahhal, Riccardo Ricci, Farid Melgani

Remote Sensing · 23.04.2024 · 16 (9) · 1477

DOI: 10.3390/rs16091477 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Remote Sensing, online erschienen am 23.04.2024. Verfasst von Yakoub Bazi und 4 weiteren Autor:innen. Die Arbeit wurde bisher 135-mal zitiert (Crossref) und nennt 67 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: King Saud University.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

In this paper, we delve into the innovative application of large language models (LLMs) and their extension, large vision-language models (LVLMs), in the field of remote sensing (RS) image analysis. We particularly emphasize their multi-tasking potential with a focus on image captioning and visual question answering (VQA). In particular, we introduce an improved version of the Large Language and Vision Assistant Model (LLaVA), specifically adapted for RS imagery through a low-rank adaptation approach. To evaluate the model performance, we create the RS-instructions dataset, a comprehensive benchmark dataset that integrates four diverse single-task datasets related to captioning and VQA. The experimental results confirm the model’s effectiveness, marking a step forward toward the development of efficient multi-task models for RS image analysis.

Förderung

  • King Saud University

Zitieren

APA

Bazi, Y., Bashmal, L., Rahhal, M. M. A., Ricci, R., & Melgani, F. (2024). RS-LLaVA: A Large Vision-Language Model for Joint Captioning and Question Answering in Remote Sensing Imagery. Remote Sensing, 16(9), 1477. https://doi.org/10.3390/rs16091477

Vancouver

Bazi Y, Bashmal L, Rahhal MMA, Ricci R, Melgani F. RS-LLaVA: A Large Vision-Language Model for Joint Captioning and Question Answering in Remote Sensing Imagery. Remote Sensing. 2024;16(9):1477. doi:10.3390/rs16091477

BibTeX
@article{Bazi2024,
  author = {Bazi, Yakoub and Bashmal, Laila and Rahhal, Mohamad Mahmoud Al and Ricci, Riccardo and Melgani, Farid},
  title = {RS-LLaVA: A Large Vision-Language Model for Joint Captioning and Question Answering in Remote Sensing Imagery},
  journal = {Remote Sensing},
  year = {2024},
  volume = {16},
  number = {9},
  pages = {1477},
  doi = {10.3390/rs16091477}
}
Kostenlos starten →