Causality-Inspired Invariant Representation Learning for Text-Based Person Retrieval
Proceedings of the AAAI Conference on Artificial Intelligence · 24.03.2024 · 38 (12) · 14052-14060
DOI: 10.1609/aaai.v38i12.29314 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 24.03.2024. Verfasst von Yu Liu und 4 weiteren Autor:innen. Die Arbeit wurde bisher 38-mal zitiert (Crossref).
Abstract (Originalsprache)
Text-based Person Retrieval (TPR) aims to retrieve relevant images of specific pedestrians based on the given textual query. The mainstream approaches primarily leverage pretrained deep neural networks to learn the mapping of visual and textual modalities into a common latent space for cross-modality matching. Despite their remarkable achievements, existing efforts mainly focus on learning the statistical cross-modality correlation found in training data, other than the intrinsic causal correlation. As a result, they often struggle to retrieve accurately in the face of environmental changes such as illumination, pose, and occlusion, or when encountering images with similar attributes. In this regard, we pioneer the observation of TPR from a causal view. Specifically, we assume that each image is composed of a mixture of causal factors (which are semantically consistent with text descriptions) and non-causal factors (retrieval-irrelevant, e.g., background), and only the former can lead to reliable retrieval judgments. Our goal is to extract text-critical robust visual representation (i.e., causal factors) and establish domain invariant cross-modality correlations for accurate and reliable retrieval. However, causal/non-causal factors are unobserved, so we emphasize that ideal causal factors that can simulate causal scenes should satisfy two basic principles:1) Independence: being independent of non-causal factors, and 2)Sufficiency: being causally sufficient for TPR across different environments. Building on that, we propose an Invariant Representation Learning method for TPR (IRLT), that enforces the visual representations to satisfy the two aforementioned critical properties. Extensive experiments on three datasets clearly demonstrate the advantages of IRLT over leading baselines in terms of accuracy and generalization.
Zitieren
APA
Liu, Y., Qin, G., Chen, H., Cheng, Z., & Yang, X. (2024). Causality-Inspired Invariant Representation Learning for Text-Based Person Retrieval. Proceedings of the AAAI Conference on Artificial Intelligence, 38(12), 14052-14060. https://doi.org/10.1609/aaai.v38i12.29314
Vancouver
Liu Y, Qin G, Chen H, Cheng Z, Yang X. Causality-Inspired Invariant Representation Learning for Text-Based Person Retrieval. Proceedings of the AAAI Conference on Artificial Intelligence. 2024;38(12):14052-14060. doi:10.1609/aaai.v38i12.29314
BibTeX
@article{Liu2024,
author = {Liu, Yu and Qin, Guihe and Chen, Haipeng and Cheng, Zhiyong and Yang, Xun},
title = {Causality-Inspired Invariant Representation Learning for Text-Based Person Retrieval},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2024},
volume = {38},
number = {12},
pages = {14052--14060},
doi = {10.1609/aaai.v38i12.29314}
}
EUVIMED