CARM: Cross-Modal Alignment Recovery for Lightweight Referring Expression Comprehension
Electronics · 07.06.2026 · 15 (12) · 2509
DOI: 10.3390/electronics15122509 ↗Kurz gefasst
Fachartikel in der Zeitschrift Electronics, online erschienen am 07.06.2026. Verfasst von Gengsheng Zheng und 4 weiteren Autor:innen. Die Arbeit nennt 40 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Plan Innovation of Hubei Province, Hubei Provincial Science and Technology Program Project, Hubei Key Laboratory of Intelligent Robot Innovation Fund.
Abstract (Originalsprache)
Referring Expression Comprehension (REC) localizes a target object in an image given a natural-language referring expression and is a core benchmark for fine-grained vision–language alignment. Recent detection-style multimodal Transformers achieve strong REC performance but typically rely on high-capacity visual and textual backbones, incurring substantial storage and compute costs. Replacing these backbones with lightweight alternatives greatly reduces model size, yet often degrades cross-modal alignment and yields a persistent accuracy gap. We propose CARM, a minimally invasive Cross-modal Alignment Recovery Module inserted between lightweight backbones and the downstream multimodal Transformer, requiring no changes to either component. CARM injects complementary priors via bidirectional prompts and uses a Cross-Attention Gate (CAG) to adaptively filter and scale prompt-induced updates; it further integrates Tree-of-Attributes Prompts (TAPs) to strengthen fine-grained grounding of attributes such as color, size, and spatial location. On RefCOCO, switching to lightweight backbones drops P@1 (IoU ≥ 0.5) to 84.51, while CARM improves it to 86.23, recovering most of the loss. Meanwhile, the overall model storage (checkpoint) is reduced by about 80%, demonstrating that the cross-modal alignment degradation induced by compression can be effectively restored.
Förderung
- Plan Innovation of Hubei Province
- Hubei Provincial Science and Technology Program Project
- Hubei Key Laboratory of Intelligent Robot Innovation Fund
Zitieren
APA
Zheng, G., Zhang, Q., Song, M., Zhang, X., & Wang, J. (2026). CARM: Cross-Modal Alignment Recovery for Lightweight Referring Expression Comprehension. Electronics, 15(12), 2509. https://doi.org/10.3390/electronics15122509
Vancouver
Zheng G, Zhang Q, Song M, Zhang X, Wang J. CARM: Cross-Modal Alignment Recovery for Lightweight Referring Expression Comprehension. Electronics. 2026;15(12):2509. doi:10.3390/electronics15122509
BibTeX
@article{Zheng2026,
author = {Zheng, Gengsheng and Zhang, Qiang and Song, Meng and Zhang, Xinghu and Wang, Jianhua},
title = {CARM: Cross-Modal Alignment Recovery for Lightweight Referring Expression Comprehension},
journal = {Electronics},
year = {2026},
volume = {15},
number = {12},
pages = {2509},
doi = {10.3390/electronics15122509}
}
EUVIMED