EUVIMEDEUVIMED entdecken
Crossref

Dynamic Modality Reweighting for Robust Vision–Language Models under Noisy Multimodal Environments

Vinuja G, Niyas Ahamed A

Turkish Journal of Engineering · 01.05.2026 · 10 (2) · 370-377

DOI: 10.31127/tuje.1813692 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Turkish Journal of Engineering, online erschienen am 01.05.2026. Verfasst von Vinuja G und 1 weiteren Autor:innen. Die Arbeit nennt 25 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Recent Vision Language Models (VLMs) perform poorly when either the visual or textual part is affected by noise like blur, occlusion, or unclear text. This paper presents a Dynamic Modality Reweighting (DMR) framework that balances the contributions of visual and textual features based on their estimated reliability during inference. The structure includes a Confidence Estimation Network (CEN) to evaluate trust scores for each modality, followed by a Dynamic Fusion Layer (DFL) that combines embeddings using data-driven weights. Experimental results on noisy versions of the MS-COCO, Flickr30k, and Visual Genome datasets show up to 23% improvement in multimodal consistency and a 17% decrease in semantic drift when compared to baseline CLIP and BLIP-2 models.

Zitieren

APA

G, V., & A, N. A. (2026). Dynamic Modality Reweighting for Robust Vision–Language Models under Noisy Multimodal Environments. Turkish Journal of Engineering, 10(2), 370-377. https://doi.org/10.31127/tuje.1813692

Vancouver

G V, A NA. Dynamic Modality Reweighting for Robust Vision–Language Models under Noisy Multimodal Environments. Turkish Journal of Engineering. 2026;10(2):370-377. doi:10.31127/tuje.1813692

BibTeX
@article{G2026,
  author = {G, Vinuja and A, Niyas Ahamed},
  title = {Dynamic Modality Reweighting for Robust Vision–Language Models under Noisy Multimodal Environments},
  journal = {Turkish Journal of Engineering},
  year = {2026},
  volume = {10},
  number = {2},
  pages = {370--377},
  doi = {10.31127/tuje.1813692}
}
Kostenlos starten →