EUVIMEDEUVIMED entdecken
Crossref

Benchmarking Multimodal Knowledge Conflict for Large Multimodal Models

Yifan Jia, Yuntao Du, Kailin Jiang, Yuyang Liang, Qihan Ren, Yi Xin, Rui Yang, Fenze Feng, MingCai Chen, Hengyang Lu, Haozhe Wang, Xiaoye Qu und 2 weitere

Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (27) · 22283-22291

DOI: 10.1609/aaai.v40i27.39385 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Yifan Jia und 13 weiteren Autor:innen. Die Arbeit wurde bisher 3-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Large Multimodal Models (LMMs) face notable challenges when encountering multimodal knowledge conflicts, particularly under retrieval-augmented generation (RAG) frameworks, where the contextual information from external sources may contradict the model’s internal parametric knowledge, leading to unreliable outputs. However, existing benchmarks fail to reflect such realistic conflict scenarios. Most focus solely on intra-memory conflicts, while context-memory and inter-context conflicts remain largely unaddressed. Furthermore, commonly used factual knowledge-based evaluations are often overlooked, and existing datasets lack a thorough investigation into conflict detection capabilities.To bridge this gap, we propose MMKC-Bench, a benchmark designed to evaluate factual knowledge conflicts in both context-memory and inter-context scenarios. MMKC-Bench encompasses four types of multimodal knowledge conflicts and includes 1,881 knowledge instances and 3,997 images across 32 broad types, collected through automated pipelines with human verification. We evaluate four representative series of LMMs on both model behavior analysis and conflict detection tasks. Our findings show that while current LMMs are capable of recognizing knowledge conflicts, they tend to favor internal parametric knowledge over external evidence. We hope MMKC-Bench will foster further research in multimodal knowledge conflict and enhance the development of multimodal RAG systems.

Zitieren

APA

Jia, Y., Du, Y., Jiang, K., Liang, Y., Ren, Q., Xin, Y., Yang, R., Feng, F., Chen, M., Lu, H., Wang, H., Qu, X., Li, Q., & Liu, D. (2026). Benchmarking Multimodal Knowledge Conflict for Large Multimodal Models. Proceedings of the AAAI Conference on Artificial Intelligence, 40(27), 22283-22291. https://doi.org/10.1609/aaai.v40i27.39385

Vancouver

Jia Y, Du Y, Jiang K, Liang Y, Ren Q, Xin Y, et al. Benchmarking Multimodal Knowledge Conflict for Large Multimodal Models. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(27):22283-22291. doi:10.1609/aaai.v40i27.39385

BibTeX
@article{Jia2026,
  author = {Jia, Yifan and Du, Yuntao and Jiang, Kailin and Liang, Yuyang and Ren, Qihan and Xin, Yi and Yang, Rui and Feng, Fenze and Chen, MingCai and Lu, Hengyang and Wang, Haozhe and Qu, Xiaoye and Li, Qian and Liu, Dongrui},
  title = {Benchmarking Multimodal Knowledge Conflict for Large Multimodal Models},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2026},
  volume = {40},
  number = {27},
  pages = {22283--22291},
  doi = {10.1609/aaai.v40i27.39385}
}
Kostenlos starten →