EUVIMEDEUVIMED entdecken
Crossref

Adaptive Uncertainty-Based Learning for Text-Based Person Retrieval

Shenshen Li, Chen He, Xing Xu, Fumin Shen, Yang Yang, Heng Tao Shen

Proceedings of the AAAI Conference on Artificial Intelligence · 24.03.2024 · 38 (4) · 3172-3180

DOI: 10.1609/aaai.v38i4.28101 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 24.03.2024. Verfasst von Shenshen Li und 5 weiteren Autor:innen. Die Arbeit wurde bisher 58-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Text-based person retrieval aims at retrieving a specific pedestrian image from a gallery based on textual descriptions. The primary challenge is how to overcome the inherent heterogeneous modality gap in the situation of significant intra-class variation and minimal inter-class variation. Existing approaches commonly employ vision-language pre-training or attention mechanisms to learn appropriate cross-modal alignments from noise inputs. Despite commendable progress, current methods inevitably suffer from two defects: 1) Matching ambiguity, which mainly derives from unreliable matching pairs; 2) One-sided cross-modal alignments, stemming from the absence of exploring one-to-many correspondence, i.e., coarse-grained semantic alignment. These critical issues significantly deteriorate retrieval performance. To this end, we propose a novel framework termed Adaptive Uncertainty-based Learning (AUL) for text-based person retrieval from the uncertainty perspective. Specifically, our AUL framework consists of three key components: 1) Uncertainty-aware Matching Filtration that leverages Subjective Logic to effectively mitigate the disturbance of unreliable matching pairs and select high-confidence cross-modal matches for training; 2) Uncertainty-based Alignment Refinement, which not only simulates coarse-grained alignments by constructing uncertainty representations but also performs progressive learning to incorporate coarse- and fine-grained alignments properly; 3) Cross-modal Masked Modeling that aims at exploring more comprehensive relations between vision and language. Extensive experiments demonstrate that our AUL method consistently achieves state-of-the-art performance on three benchmark datasets in supervised, weakly supervised, and domain generalization settings. Our code is available at https://github.com/CFM-MSG/Code-AUL.

Zitieren

APA

Li, S., He, C., Xu, X., Shen, F., Yang, Y., & Shen, H. T. (2024). Adaptive Uncertainty-Based Learning for Text-Based Person Retrieval. Proceedings of the AAAI Conference on Artificial Intelligence, 38(4), 3172-3180. https://doi.org/10.1609/aaai.v38i4.28101

Vancouver

Li S, He C, Xu X, Shen F, Yang Y, Shen HT. Adaptive Uncertainty-Based Learning for Text-Based Person Retrieval. Proceedings of the AAAI Conference on Artificial Intelligence. 2024;38(4):3172-3180. doi:10.1609/aaai.v38i4.28101

BibTeX
@article{Li2024,
  author = {Li, Shenshen and He, Chen and Xu, Xing and Shen, Fumin and Yang, Yang and Shen, Heng Tao},
  title = {Adaptive Uncertainty-Based Learning for Text-Based Person Retrieval},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2024},
  volume = {38},
  number = {4},
  pages = {3172--3180},
  doi = {10.1609/aaai.v38i4.28101}
}
Kostenlos starten →