EUVIMEDEUVIMED entdecken
Crossref

Fine-Grained Image Captioning by Ranking Diffusion Transformer

Jun Wan, Min Gan, Lefei Zhang, Jie Zhou, Jun Liu, Bo Du, C L Philip Chen

IEEE Transactions on Image Processing · 2025 · 34 · 8332-8344

DOI: 10.1109/tip.2025.3641303 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Image Processing, 2025. Verfasst von Jun Wan und 6 weiteren Autor:innen. Die Arbeit wurde bisher 8-mal zitiert (Crossref) und nennt 65 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Natural Science Foundation of Hubei Province.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The CLIP visual feature-based image captioning models have developed rapidly and achieved remarkable results. However, existing models still struggle to produce descriptive and discriminative captions because they insufficiently exploit fine-grained visual cues and fail to model complex vision-language alignment. To address these limitations, we propose a Ranking Diffusion Transformer (RDT), which integrates a Ranking Visual Encoder (RVE) and a Ranking Loss (RL) for fine-grained image captioning. The RVE introduces a novel ranking attention mechanism that effectively mines diverse and discriminative visual information from CLIP features. Meanwhile, the RL leverages the ranking of generated caption quality as a global semantic supervisory signal, thereby enhancing the diffusion process and strengthening vision-language semantic alignment. We show that by collaborating RVE and RL via the novel RDT-and by gradually adding and removing noise in the diffusion process-more discriminative visual features are learned and precisely aligned with the language features. Experimental results on popular benchmark datasets demonstrate that our proposed RDT surpasses existing state-of-the-art image captioning models in the literature. The code is publicly available at: https://github.com/junwan2014/RDT.

Förderung

  • National Natural Science Foundation of China
  • Natural Science Foundation of Hubei Province

Zitieren

APA

Wan, J., Gan, M., Zhang, L., Zhou, J., Liu, J., Du, B., & Chen, C. L. P. (2025). Fine-Grained Image Captioning by Ranking Diffusion Transformer. IEEE Transactions on Image Processing, 34, 8332-8344. https://doi.org/10.1109/tip.2025.3641303

Vancouver

Wan J, Gan M, Zhang L, Zhou J, Liu J, Du B, et al. Fine-Grained Image Captioning by Ranking Diffusion Transformer. IEEE Transactions on Image Processing. 2025;34:8332-8344. doi:10.1109/tip.2025.3641303

BibTeX
@article{Wan2025,
  author = {Wan, Jun and Gan, Min and Zhang, Lefei and Zhou, Jie and Liu, Jun and Du, Bo and Chen, C L Philip},
  title = {Fine-Grained Image Captioning by Ranking Diffusion Transformer},
  journal = {IEEE Transactions on Image Processing},
  year = {2025},
  volume = {34},
  pages = {8332--8344},
  doi = {10.1109/tip.2025.3641303}
}
Kostenlos starten →