Text-Guided Vision Token Reduction With Low-Rank Adaptation for Efficient Visual Grounding
IEEE Transactions on Circuits and Systems for Video Technology · 2026-08 · 36 (8) · 11391-11401
DOI: 10.1109/tcsvt.2026.3684576 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Circuits and Systems for Video Technology, 2026-08. Verfasst von Liangtao Shi und 5 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 64 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Key Scientific and Technological Research Project of Hunan Province, China,.
Abstract (Originalsprache)
Transformer-based pretrained models have advanced Visual Grounding (VG) significantly, but their scaling up has caused soaring training and inference costs. While efforts adapting Parameter-Efficient Fine-Tuning to VG have cut training expenses to some extent, inference costs are overlooked, stemming from Transformer’s computation costs growing quadratically with input token length. To address the above challenge, we propose a text-guided token prune method based on a one-stream architecture for VG, named OneSVG. Specifically, OneSVG transfers low-rank adaptation LoRA to VG, and calculates the relevance between each vision token and text semantics in multiple stages, and gradually prunes the vision tokens with low relevance. In addition, traditional VG methods use a single [REG] token to predict bounding boxes, and the [REG] token relies on complete vision tokens during training, pruning large number of vision tokens will disrupt the spatial orientation information of images. Therefore, the prediction head in OneSVG first restores the square structure of images by padding the missing area, and then accepts complete vision tokens. Experimental results on three widely-used benchmarks demonstrate that our OneSVG achieves state-of-the-art real-time speed while maintaining the best accuracy with only 34.3% tokens. Codes and models are available at https://github.com/ltShi/OneSVG
Förderung
- National Natural Science Foundation of China
- Key Scientific and Technological Research Project of Hunan Province, China,
Zitieren
APA
Shi, L., Liu, T., Xie, J., Li, N., Zhong, B., & Hong, R. (2026). Text-Guided Vision Token Reduction With Low-Rank Adaptation for Efficient Visual Grounding. IEEE Transactions on Circuits and Systems for Video Technology, 36(8), 11391-11401. https://doi.org/10.1109/tcsvt.2026.3684576
Vancouver
Shi L, Liu T, Xie J, Li N, Zhong B, Hong R. Text-Guided Vision Token Reduction With Low-Rank Adaptation for Efficient Visual Grounding. IEEE Transactions on Circuits and Systems for Video Technology. 2026;36(8):11391-11401. doi:10.1109/tcsvt.2026.3684576
BibTeX
@article{Shi2026,
author = {Shi, Liangtao and Liu, Ting and Xie, Jinxia and Li, Ning and Zhong, Bineng and Hong, Richang},
title = {Text-Guided Vision Token Reduction With Low-Rank Adaptation for Efficient Visual Grounding},
journal = {IEEE Transactions on Circuits and Systems for Video Technology},
year = {2026},
volume = {36},
number = {8},
pages = {11391--11401},
doi = {10.1109/tcsvt.2026.3684576}
}
EUVIMED