EUVIMEDEUVIMED entdecken
Crossref

MDK12-Bench: A Multi-Discipline Benchmark for Evaluating Reasoning in Multimodal Large Language Models

Pengfei Zhou, Xiaopeng Peng, Fanrui Zhang, Zhaopan Xu, Jiaxin Ai, Yansheng Qiu, Wangbo Zhao, Jiajun Song, Chuanhao Li, Weidong Tang, Zhen Li, Haoquan Zhang und 9 weitere

Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (34) · 28982-28990

DOI: 10.1609/aaai.v40i34.40134 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Pengfei Zhou und 20 weiteren Autor:innen. Die Arbeit wurde bisher 4-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Multimodal large language models (MLLMs), which integrate language and visual cues for problem-solving, are crucial for advancing artificial general intelligence (AGI). However, current benchmarks for measuring the intelligence of MLLMs suffer from limited scale, narrow coverage, and unstructured knowledge, offering only static and undifferentiated evaluations. To bridge this gap, we introduce MDK12-Bench, a large-scale multidisciplinary benchmark built from real-world K–12 exams spanning six disciplines with 141K instances and 6,225 knowledge points organized in a six-layer taxonomy. Covering five question formats with difficulty and year annotations, it enables comprehensive evaluation to capture the extent to which MLLMs perform over four dimensions: 1) difficulty levels, 2) temporal (cross-year) shifts, 3) contextual shifts, and 4) knowledge-driven reasoning. We propose a novel dynamic evaluation framework that introduces unfamiliar visual, textual, and question form shifts to challenge model generalization while improving benchmark objectivity and longevity by mitigating data contamination. We further evaluate knowledge-point reference-augmented generation (KP-RAG) to examine the role of knowledge in reasoning. Key findings reveal limitations in current MLLMs in multiple aspects and provide guidance for enhancing model reasoning, robustness, and AI-assisted education.

Zitieren

APA

Zhou, P., Peng, X., Zhang, F., Xu, Z., Ai, J., Qiu, Y., Zhao, W., Song, J., Li, C., Tang, W., Li, Z., Zhang, H., Li, Z., Mao, X., Feng, Y., Sun, J., Wang, K., Chang, X., Shao, W., …, & Zhang, K. (2026). MDK12-Bench: A Multi-Discipline Benchmark for Evaluating Reasoning in Multimodal Large Language Models. Proceedings of the AAAI Conference on Artificial Intelligence, 40(34), 28982-28990. https://doi.org/10.1609/aaai.v40i34.40134

Vancouver

Zhou P, Peng X, Zhang F, Xu Z, Ai J, Qiu Y, et al. MDK12-Bench: A Multi-Discipline Benchmark for Evaluating Reasoning in Multimodal Large Language Models. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(34):28982-28990. doi:10.1609/aaai.v40i34.40134

BibTeX
@article{Zhou2026,
  author = {Zhou, Pengfei and Peng, Xiaopeng and Zhang, Fanrui and Xu, Zhaopan and Ai, Jiaxin and Qiu, Yansheng and Zhao, Wangbo and Song, Jiajun and Li, Chuanhao and Tang, Weidong and Li, Zhen and Zhang, Haoquan and Li, Zizhen and Mao, Xiaofeng and Feng, Yukang and Sun, Jianwen and Wang, Kai and Chang, Xiaojun and Shao, Wenqi and You, Yang and Zhang, Kaipeng},
  title = {MDK12-Bench: A Multi-Discipline Benchmark for Evaluating Reasoning in Multimodal Large Language Models},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2026},
  volume = {40},
  number = {34},
  pages = {28982--28990},
  doi = {10.1609/aaai.v40i34.40134}
}
Kostenlos starten →