EUVIMEDEUVIMED entdecken
Crossref

R-Bench: Are Your Large Multimodal Model Robust to Real-World Corruptions?

Chunyi Li, Jianbo Zhang, Zicheng Zhang, Haoning Wu, Yuan Tian, Wei Sun, Guo Lu, Xiongkuo Min, Xiaohong Liu, Weisi Lin, Xiao-Ping Zhang, Guangtao Zhai

IEEE Journal of Selected Topics in Signal Processing · 2025-10 · 19 (7) · 1349-1361

DOI: 10.1109/jstsp.2025.3558652 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Journal of Selected Topics in Signal Processing, 2025-10. Verfasst von Chunyi Li und 11 weiteren Autor:innen. Die Arbeit wurde bisher 11-mal zitiert (Crossref) und nennt 56 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Sichuan Science and Technology Program.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The outstanding performance of Large Multimodal Models (LMMs) has made them widely applied in vision-related tasks. However, various corruptions in the real world mean that images will not be as ideal as in simulations, presenting significant challenges for the practical application of LMMs. To address this issue, we introduce R-Bench, a benchmark focused on the Real-world Robustness of LMMs. Specifically, we: (a) model the complete link from user capture to LMMs reception, comprising 33 corruption dimensions, including 7 steps according to the corruption sequence, and 7 groups based on low-level attributes; (b) collect reference/distorted image dataset before/after corruption, including 2,970 question-answer pairs with human labeling; (c) propose comprehensive evaluation for absolute/relative robustness and benchmark 20 mainstream LMMs. Results show that while LMMs can correctly handle the original reference images, their performance is not stable when faced with distorted images, and there is a significant gap in robustness compared to the human visual system. We hope that R-Bench will inspire improving the robustness of LMMs, extending them from experimental simulations to the real-world application.

Förderung

  • National Natural Science Foundation of China
  • Sichuan Science and Technology Program

Zitieren

APA

Li, C., Zhang, J., Zhang, Z., Wu, H., Tian, Y., Sun, W., Lu, G., Min, X., Liu, X., Lin, W., Zhang, X., & Zhai, G. (2025). R-Bench: Are Your Large Multimodal Model Robust to Real-World Corruptions?. IEEE Journal of Selected Topics in Signal Processing, 19(7), 1349-1361. https://doi.org/10.1109/jstsp.2025.3558652

Vancouver

Li C, Zhang J, Zhang Z, Wu H, Tian Y, Sun W, et al. R-Bench: Are Your Large Multimodal Model Robust to Real-World Corruptions?. IEEE Journal of Selected Topics in Signal Processing. 2025;19(7):1349-1361. doi:10.1109/jstsp.2025.3558652

BibTeX
@article{Li2025,
  author = {Li, Chunyi and Zhang, Jianbo and Zhang, Zicheng and Wu, Haoning and Tian, Yuan and Sun, Wei and Lu, Guo and Min, Xiongkuo and Liu, Xiaohong and Lin, Weisi and Zhang, Xiao-Ping and Zhai, Guangtao},
  title = {R-Bench: Are Your Large Multimodal Model Robust to Real-World Corruptions?},
  journal = {IEEE Journal of Selected Topics in Signal Processing},
  year = {2025},
  volume = {19},
  number = {7},
  pages = {1349--1361},
  doi = {10.1109/jstsp.2025.3558652}
}
Kostenlos starten →