EUVIMEDEUVIMED entdecken
Crossref

FineG-RAG: Fine-Grained Retrieval-Augmented Generation for Multimodal Large Language Models

Lu Jin, Cheng Lin, Xinguang Xiang, Yanpeng Sun, Zechao Li, Jinhui Tang

IEEE Transactions on Circuits and Systems for Video Technology · 2026-06 · 36 (6) · 8675-8686

DOI: 10.1109/tcsvt.2026.3686876 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Circuits and Systems for Video Technology, 2026-06. Verfasst von Lu Jin und 5 weiteren Autor:innen. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 53 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Major Science and Technology Program of Jiangsu Province, Basic Research Program of Jiangsu Province.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Fine-grained visual recognition refers to the ability to distinguish subtle differences between visually similar objects—a fundamental yet challenging capability for Multimodal Large Language Models (MLLMs). In this paper, we observe that even strong open-source MLLMs, such as Qwen2-VL and InternVL2, still struggle with accurately identifying fine-grained categories. These models often fail to attend to subtle but critical details for precise discrimination. To unlock this potential, we propose FineG-RAG, a retrieval-augmented generation pipeline designed to enhance the fine-grained recognition capabilities of MLLMs. FineG-RAG integrates external fine-grained knowledge into the recognition process via a generalized retriever. To support this, we construct fine-grained visual-language knowledge database containing representative images with wide visual diversity and expert-crafted attribute descriptions from multiple perspectives. Relevant fine-grained knowledge is retrieved from this database and fed into a visual-language augmented prompt, which provides rich multimodal context to guide MLLMs in generating accurate labels. To better evaluate the fine-grained recognition capabilities of MLLMs, we design a multiple-choice evaluation strategy based on publicly four fine-grained datasets. Extensive experiments demonstrate that FineG-RAG consistently outperforms baseline methods, achieving superior recognition accuracy across a range of off-the-shelf, open-source MLLMs.

Förderung

  • National Natural Science Foundation of China
  • Major Science and Technology Program of Jiangsu Province
  • Basic Research Program of Jiangsu Province

Zitieren

APA

Jin, L., Lin, C., Xiang, X., Sun, Y., Li, Z., & Tang, J. (2026). FineG-RAG: Fine-Grained Retrieval-Augmented Generation for Multimodal Large Language Models. IEEE Transactions on Circuits and Systems for Video Technology, 36(6), 8675-8686. https://doi.org/10.1109/tcsvt.2026.3686876

Vancouver

Jin L, Lin C, Xiang X, Sun Y, Li Z, Tang J. FineG-RAG: Fine-Grained Retrieval-Augmented Generation for Multimodal Large Language Models. IEEE Transactions on Circuits and Systems for Video Technology. 2026;36(6):8675-8686. doi:10.1109/tcsvt.2026.3686876

BibTeX
@article{Jin2026,
  author = {Jin, Lu and Lin, Cheng and Xiang, Xinguang and Sun, Yanpeng and Li, Zechao and Tang, Jinhui},
  title = {FineG-RAG: Fine-Grained Retrieval-Augmented Generation for Multimodal Large Language Models},
  journal = {IEEE Transactions on Circuits and Systems for Video Technology},
  year = {2026},
  volume = {36},
  number = {6},
  pages = {8675--8686},
  doi = {10.1109/tcsvt.2026.3686876}
}
Kostenlos starten →