EUVIMEDEUVIMED entdecken
Crossref

CrossCheck-Bench: Diagnosing Compositional Failures in Multimodal Conflict Resolution

Baoliang Tian, Yuxuan Si, Jilong Wang, LingYao Li, Zhongyuan Bao, Zineng Zhou, Tao Wang, Sixu Li, Ziyao Xu, Mingze Wang, Zhouzhuo Zhang, Zhihao Wang und 4 weitere

Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (31) · 25887-25895

DOI: 10.1609/aaai.v40i31.39788 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Baoliang Tian und 15 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Multimodal Large Language Models are primarily trained and evaluated on aligned image-text pairs, which leaves their ability to detect and resolve real-world inconsistencies largely unexplored. In open-domain applications visual and textual cues often conflict, requiring models to perform structured reasoning beyond surface-level alignment. We introduce CrossCheck-Bench, a diagnostic benchmark for evaluating contradiction detection in multimodal inputs. The benchmark adopts a hierarchical task framework covering three levels of reasoning complexity and defines seven atomic capabilities essential for resolving cross-modal inconsistencies. CrossCheck-Bench includes 15k question-answer pairs sourced from real-world artifacts with synthetically injected contradictions. The dataset is constructed through a multi-stage annotation pipeline involving more than 450 expert hours to ensure semantic validity and calibrated difficulty across perception, integration, and reasoning. We evaluate 13 state-of-the-art vision-language models and observe a consistent performance drop as tasks shift from perceptual matching to logical contradiction detection. Most models perform well on isolated entity recognition but fail when multiple clues must be synthesized for conflict reasoning. Capability-level analysis further reveals uneven skill acquisition, especially in tasks requiring multi-step inference or rule-based validation. Additional probing shows that conventional prompting strategies such as Chain-of-Thought and Set-of-Mark yield only marginal gains. By contrast, methods that interleave symbolic reasoning with grounded visual processing achieve more stable improvements. These results highlight a persistent bottleneck in multimodal reasoning and suggest new directions for building models capable of robust cross-modal verification.

Zitieren

APA

Tian, B., Si, Y., Wang, J., Li, L., Bao, Z., Zhou, Z., Wang, T., Li, S., Xu, Z., Wang, M., Zhang, Z., Wang, Z., Yun, Y. K., Tian, K., Yang, N., & Qiu, M. (2026). CrossCheck-Bench: Diagnosing Compositional Failures in Multimodal Conflict Resolution. Proceedings of the AAAI Conference on Artificial Intelligence, 40(31), 25887-25895. https://doi.org/10.1609/aaai.v40i31.39788

Vancouver

Tian B, Si Y, Wang J, Li L, Bao Z, Zhou Z, et al. CrossCheck-Bench: Diagnosing Compositional Failures in Multimodal Conflict Resolution. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(31):25887-25895. doi:10.1609/aaai.v40i31.39788

BibTeX
@article{Tian2026,
  author = {Tian, Baoliang and Si, Yuxuan and Wang, Jilong and Li, LingYao and Bao, Zhongyuan and Zhou, Zineng and Wang, Tao and Li, Sixu and Xu, Ziyao and Wang, Mingze and Zhang, Zhouzhuo and Wang, Zhihao and Yun, Yi Ke and Tian, Ke and Yang, Ning and Qiu, Minghui},
  title = {CrossCheck-Bench: Diagnosing Compositional Failures in Multimodal Conflict Resolution},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2026},
  volume = {40},
  number = {31},
  pages = {25887--25895},
  doi = {10.1609/aaai.v40i31.39788}
}
Kostenlos starten →