EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

CARM: Cross-Modal Alignment Recovery for Lightweight Referring Expression Comprehension

Gengsheng Zheng, Qiang Zhang, Meng Song, Xinghu Zhang, Jianhua Wang

Electronics · 07.06.2026 · 15 (12) · 2509

DOI: 10.3390/electronics15122509 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Electronics, online erschienen am 07.06.2026. Verfasst von Gengsheng Zheng und 4 weiteren Autor:innen. Die Arbeit nennt 40 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Plan Innovation of Hubei Province, Hubei Provincial Science and Technology Program Project, Hubei Key Laboratory of Intelligent Robot Innovation Fund.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Referring Expression Comprehension (REC) localizes a target object in an image given a natural-language referring expression and is a core benchmark for fine-grained vision–language alignment. Recent detection-style multimodal Transformers achieve strong REC performance but typically rely on high-capacity visual and textual backbones, incurring substantial storage and compute costs. Replacing these backbones with lightweight alternatives greatly reduces model size, yet often degrades cross-modal alignment and yields a persistent accuracy gap. We propose CARM, a minimally invasive Cross-modal Alignment Recovery Module inserted between lightweight backbones and the downstream multimodal Transformer, requiring no changes to either component. CARM injects complementary priors via bidirectional prompts and uses a Cross-Attention Gate (CAG) to adaptively filter and scale prompt-induced updates; it further integrates Tree-of-Attributes Prompts (TAPs) to strengthen fine-grained grounding of attributes such as color, size, and spatial location. On RefCOCO, switching to lightweight backbones drops P@1 (IoU ≥ 0.5) to 84.51, while CARM improves it to 86.23, recovering most of the loss. Meanwhile, the overall model storage (checkpoint) is reduced by about 80%, demonstrating that the cross-modal alignment degradation induced by compression can be effectively restored.

Förderung

  • Plan Innovation of Hubei Province
  • Hubei Provincial Science and Technology Program Project
  • Hubei Key Laboratory of Intelligent Robot Innovation Fund

Zitieren

APA

Zheng, G., Zhang, Q., Song, M., Zhang, X., & Wang, J. (2026). CARM: Cross-Modal Alignment Recovery for Lightweight Referring Expression Comprehension. Electronics, 15(12), 2509. https://doi.org/10.3390/electronics15122509

Vancouver

Zheng G, Zhang Q, Song M, Zhang X, Wang J. CARM: Cross-Modal Alignment Recovery for Lightweight Referring Expression Comprehension. Electronics. 2026;15(12):2509. doi:10.3390/electronics15122509

BibTeX
@article{Zheng2026,
  author = {Zheng, Gengsheng and Zhang, Qiang and Song, Meng and Zhang, Xinghu and Wang, Jianhua},
  title = {CARM: Cross-Modal Alignment Recovery for Lightweight Referring Expression Comprehension},
  journal = {Electronics},
  year = {2026},
  volume = {15},
  number = {12},
  pages = {2509},
  doi = {10.3390/electronics15122509}
}
Kostenlos starten →