Dynamic Modality Reweighting for Robust Vision–Language Models under Noisy Multimodal Environments
Turkish Journal of Engineering · 01.05.2026 · 10 (2) · 370-377
DOI: 10.31127/tuje.1813692 ↗Kurz gefasst
Fachartikel in der Zeitschrift Turkish Journal of Engineering, online erschienen am 01.05.2026. Verfasst von Vinuja G und 1 weiteren Autor:innen. Die Arbeit nennt 25 Literaturangaben.
Abstract (Originalsprache)
Recent Vision Language Models (VLMs) perform poorly when either the visual or textual part is affected by noise like blur, occlusion, or unclear text. This paper presents a Dynamic Modality Reweighting (DMR) framework that balances the contributions of visual and textual features based on their estimated reliability during inference. The structure includes a Confidence Estimation Network (CEN) to evaluate trust scores for each modality, followed by a Dynamic Fusion Layer (DFL) that combines embeddings using data-driven weights. Experimental results on noisy versions of the MS-COCO, Flickr30k, and Visual Genome datasets show up to 23% improvement in multimodal consistency and a 17% decrease in semantic drift when compared to baseline CLIP and BLIP-2 models.
Zitieren
APA
G, V., & A, N. A. (2026). Dynamic Modality Reweighting for Robust Vision–Language Models under Noisy Multimodal Environments. Turkish Journal of Engineering, 10(2), 370-377. https://doi.org/10.31127/tuje.1813692
Vancouver
G V, A NA. Dynamic Modality Reweighting for Robust Vision–Language Models under Noisy Multimodal Environments. Turkish Journal of Engineering. 2026;10(2):370-377. doi:10.31127/tuje.1813692
BibTeX
@article{G2026,
author = {G, Vinuja and A, Niyas Ahamed},
title = {Dynamic Modality Reweighting for Robust Vision–Language Models under Noisy Multimodal Environments},
journal = {Turkish Journal of Engineering},
year = {2026},
volume = {10},
number = {2},
pages = {370--377},
doi = {10.31127/tuje.1813692}
}
EUVIMED