FineG-RAG: Fine-Grained Retrieval-Augmented Generation for Multimodal Large Language Models
IEEE Transactions on Circuits and Systems for Video Technology · 2026-06 · 36 (6) · 8675-8686
DOI: 10.1109/tcsvt.2026.3686876 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Circuits and Systems for Video Technology, 2026-06. Verfasst von Lu Jin und 5 weiteren Autor:innen. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 53 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Major Science and Technology Program of Jiangsu Province, Basic Research Program of Jiangsu Province.
Abstract (Originalsprache)
Fine-grained visual recognition refers to the ability to distinguish subtle differences between visually similar objects—a fundamental yet challenging capability for Multimodal Large Language Models (MLLMs). In this paper, we observe that even strong open-source MLLMs, such as Qwen2-VL and InternVL2, still struggle with accurately identifying fine-grained categories. These models often fail to attend to subtle but critical details for precise discrimination. To unlock this potential, we propose FineG-RAG, a retrieval-augmented generation pipeline designed to enhance the fine-grained recognition capabilities of MLLMs. FineG-RAG integrates external fine-grained knowledge into the recognition process via a generalized retriever. To support this, we construct fine-grained visual-language knowledge database containing representative images with wide visual diversity and expert-crafted attribute descriptions from multiple perspectives. Relevant fine-grained knowledge is retrieved from this database and fed into a visual-language augmented prompt, which provides rich multimodal context to guide MLLMs in generating accurate labels. To better evaluate the fine-grained recognition capabilities of MLLMs, we design a multiple-choice evaluation strategy based on publicly four fine-grained datasets. Extensive experiments demonstrate that FineG-RAG consistently outperforms baseline methods, achieving superior recognition accuracy across a range of off-the-shelf, open-source MLLMs.
Förderung
- National Natural Science Foundation of China
- Major Science and Technology Program of Jiangsu Province
- Basic Research Program of Jiangsu Province
Zitieren
APA
Jin, L., Lin, C., Xiang, X., Sun, Y., Li, Z., & Tang, J. (2026). FineG-RAG: Fine-Grained Retrieval-Augmented Generation for Multimodal Large Language Models. IEEE Transactions on Circuits and Systems for Video Technology, 36(6), 8675-8686. https://doi.org/10.1109/tcsvt.2026.3686876
Vancouver
Jin L, Lin C, Xiang X, Sun Y, Li Z, Tang J. FineG-RAG: Fine-Grained Retrieval-Augmented Generation for Multimodal Large Language Models. IEEE Transactions on Circuits and Systems for Video Technology. 2026;36(6):8675-8686. doi:10.1109/tcsvt.2026.3686876
BibTeX
@article{Jin2026,
author = {Jin, Lu and Lin, Cheng and Xiang, Xinguang and Sun, Yanpeng and Li, Zechao and Tang, Jinhui},
title = {FineG-RAG: Fine-Grained Retrieval-Augmented Generation for Multimodal Large Language Models},
journal = {IEEE Transactions on Circuits and Systems for Video Technology},
year = {2026},
volume = {36},
number = {6},
pages = {8675--8686},
doi = {10.1109/tcsvt.2026.3686876}
}
EUVIMED