EUVIMEDEUVIMED entdecken
Crossref

AMITA: Attribute-Guided Masked Image-Text Alignment for Multi-Label Image Representation

Jinyi Fang, Bingke Zhu, Jingling Yuan, Yingying Chen, Ming Tang, Jinqiao Wang

IEEE Transactions on Circuits and Systems for Video Technology · 2025-11 · 35 (11) · 11432-11447

DOI: 10.1109/tcsvt.2025.3577277 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Circuits and Systems for Video Technology, 2025-11. Verfasst von Jinyi Fang und 5 weiteren Autor:innen. Die Arbeit wurde bisher 4-mal zitiert (Crossref) und nennt 80 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Hubei Provincial Undergraduate Higher Education Provincial Level Teaching Reform Research Project.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Multi-label image classification, which involves recognizing multiple objects within a single image, is a fundamental task in computer vision. Recently, Visual-Language Models (VLMs) have made remarkable progress in this area. Many approaches combine textual and visual modalities to understand the entire image. In this paper, we find that there is a direct correlation between the accurate localization of objects and the accuracy of multi-label classification. However, previous research methods did not specifically address localization accuracy, resulting in sub-optimal accuracy. Therefore, we propose the AMITA, namely Attribute-guided Masked Image-Text Alignment for multi-label image representation. AMITA improves localization accuracy by segmenting object masks, thereby enhancing the accuracy of multi-label image classification. Additionally, AMITA introduces an AutoFocus method to handle the localization problem of small objects. AutoFocus conducts recognition by resizing and cropping the image respectively, and automatically selects the images useful for the classification target. Moreover, AMITA incorporates Attribute-guided Prompting to strengthen the semantic distinction among different categories. It uses large language models to obtain the attributes of different categories and carefully designs prompts to enhance the attribute differences among different categories. Finally, extensive experiments on three popular datasets, including MS-COCO, Pascal VOC 2007, and NUS-WIDE, demonstrate the superiority of AMITA.

Förderung

  • National Natural Science Foundation of China
  • Hubei Provincial Undergraduate Higher Education Provincial Level Teaching Reform Research Project

Zitieren

APA

Fang, J., Zhu, B., Yuan, J., Chen, Y., Tang, M., & Wang, J. (2025). AMITA: Attribute-Guided Masked Image-Text Alignment for Multi-Label Image Representation. IEEE Transactions on Circuits and Systems for Video Technology, 35(11), 11432-11447. https://doi.org/10.1109/tcsvt.2025.3577277

Vancouver

Fang J, Zhu B, Yuan J, Chen Y, Tang M, Wang J. AMITA: Attribute-Guided Masked Image-Text Alignment for Multi-Label Image Representation. IEEE Transactions on Circuits and Systems for Video Technology. 2025;35(11):11432-11447. doi:10.1109/tcsvt.2025.3577277

BibTeX
@article{Fang2025,
  author = {Fang, Jinyi and Zhu, Bingke and Yuan, Jingling and Chen, Yingying and Tang, Ming and Wang, Jinqiao},
  title = {AMITA: Attribute-Guided Masked Image-Text Alignment for Multi-Label Image Representation},
  journal = {IEEE Transactions on Circuits and Systems for Video Technology},
  year = {2025},
  volume = {35},
  number = {11},
  pages = {11432--11447},
  doi = {10.1109/tcsvt.2025.3577277}
}
Kostenlos starten →