VLCounter: Text-Aware Visual Representation for Zero-Shot Object Counting
Proceedings of the AAAI Conference on Artificial Intelligence · 24.03.2024 · 38 (3) · 2714-2722
DOI: 10.1609/aaai.v38i3.28050 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 24.03.2024. Verfasst von Seunggu Kang und 3 weiteren Autor:innen. Die Arbeit wurde bisher 53-mal zitiert (Crossref).
Abstract (Originalsprache)
Zero-Shot Object Counting~(ZSOC) aims to count referred instances of arbitrary classes in a query image without human-annotated exemplars. To deal with ZSOC, preceding studies proposed a two-stage pipeline: discovering exemplars and counting. However, there remains a challenge of vulnerability to error propagation of the sequentially designed two-stage process. In this work, we propose an one-stage baseline, Visual-Language Baseline (VLBase), exploring the implicit association of the semantic-patch embeddings of CLIP. Subsequently, we extend the VLBase to Visual-language Counter (VLCounter) by incorporating three modules devised to tailor VLBase for object counting. First, we introduce Semantic-conditioned Prompt Tuning (SPT) within the image encoder to acquire target-highlighted representations. Second, Learnable Affine Transformation (LAT) is employed to translate the semantic-patch similarity map to be appropriate for the counting task. Lastly, we transfer the layer-wisely encoded features to the decoder through Segment-aware Skip Connection (SaSC) to keep the generalization capability for unseen classes. Through extensive experiments on FSC147, CARPK, and PUCPR+, we demonstrate the benefits of our end-to-end framework, VLCounter. Code is available at https://github.com/seunggu0305/VLCounter
Zitieren
APA
Kang, S., Moon, W., Kim, E., & Heo, J. (2024). VLCounter: Text-Aware Visual Representation for Zero-Shot Object Counting. Proceedings of the AAAI Conference on Artificial Intelligence, 38(3), 2714-2722. https://doi.org/10.1609/aaai.v38i3.28050
Vancouver
Kang S, Moon W, Kim E, Heo J. VLCounter: Text-Aware Visual Representation for Zero-Shot Object Counting. Proceedings of the AAAI Conference on Artificial Intelligence. 2024;38(3):2714-2722. doi:10.1609/aaai.v38i3.28050
BibTeX
@article{Kang2024,
author = {Kang, Seunggu and Moon, WonJun and Kim, Euiyeon and Heo, Jae-Pil},
title = {VLCounter: Text-Aware Visual Representation for Zero-Shot Object Counting},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2024},
volume = {38},
number = {3},
pages = {2714--2722},
doi = {10.1609/aaai.v38i3.28050}
}
EUVIMED