EUVIMEDEUVIMED entdecken
Crossref

MHB: Medical Hallucination Benchmark for Large Language Models in Complex Clinical Tasks

Jianrong Lu, Junwei Liu, Xingyun Zheng, Minghui Yang, Jian Wang, Ping Wang, Yechao Zhang

Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (45) · 38971-38978

DOI: 10.1609/aaai.v40i45.41243 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Jianrong Lu und 6 weiteren Autor:innen. Die Arbeit wurde bisher 2-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The integration of Large Language Models (LLMs) into clinical applications presents transformative potential but is undermined by the critical risk of hallucination, the generation of plausible but factually incorrect information. Such failures pose a direct threat to patient safety and the integrity of clinical decision-making. To address this challenge, we introduce MHB, a novel and comprehensive benchmark framework designed to evaluate LLM reliability in two complex, high-stakes clinical contexts: multi-turn medical dialogues and clinical case report analysis. The core of our contribution is a systematic methodology for generating adversarial test cases by injecting ``hallucination traps" into realistic medical data, guided by a fine-grained taxonomy of clinical errors. MHB, comprising 4,695 samples and 20,288 evaluation rubrics, underwent a rigorous, two-stage validation by a panel of 60 licensed physicians from top-tier hospitals, ensuring high clinical realism and consistency. This comprehensive assessment of leading LLMs revealed significant, clinically relevant shortcomings across the board. Even the best-performing model, Claude-4-Sonnet, exhibited a hallucination rate of 29.1%, with some open-source models exceeding 57.0%. All models struggled with specific traps, like fabricated medical data or non-existent guidelines, highlighting prevalent systemic weaknesses.

Zitieren

APA

Lu, J., Liu, J., Zheng, X., Yang, M., Wang, J., Wang, P., & Zhang, Y. (2026). MHB: Medical Hallucination Benchmark for Large Language Models in Complex Clinical Tasks. Proceedings of the AAAI Conference on Artificial Intelligence, 40(45), 38971-38978. https://doi.org/10.1609/aaai.v40i45.41243

Vancouver

Lu J, Liu J, Zheng X, Yang M, Wang J, Wang P, et al. MHB: Medical Hallucination Benchmark for Large Language Models in Complex Clinical Tasks. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(45):38971-38978. doi:10.1609/aaai.v40i45.41243

BibTeX
@article{Lu2026,
  author = {Lu, Jianrong and Liu, Junwei and Zheng, Xingyun and Yang, Minghui and Wang, Jian and Wang, Ping and Zhang, Yechao},
  title = {MHB: Medical Hallucination Benchmark for Large Language Models in Complex Clinical Tasks},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2026},
  volume = {40},
  number = {45},
  pages = {38971--38978},
  doi = {10.1609/aaai.v40i45.41243}
}
Kostenlos starten →