EUVIMEDEUVIMED entdecken
Übersichtsarbeit

GynMedEval : A comprehensive dataset for evaluating the diagnostic capability of large language models in gynecology

Fangchen Dong, HanZhi Zhang, Yi Ren, Weibin Li, Yuan Li, Guangyu Cui, Peixuan Liu

International Journal of Gynecology & Obstetrics · 2026

DOI: 10.1002/ijgo.70437 ↗

Kurz gefasst

Übersichtsarbeit in der Zeitschrift International Journal of Gynecology & Obstetrics, 2026. Verfasst von Fangchen Dong und 6 weiteren Autor:innen. Beteiligte Einrichtung: School of Communications and Information Engineering, North University of China, Taiyuan, China. Hauptthemen laut MEDLINE: Gynecology, Language, Genital Diseases, Female. Die Arbeit nennt 23 Literaturangaben.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

In recent years, large language models (LLMs) have achieved significant breakthroughs across various fields, demonstrating immense potential in the medical domain. However, existing studies fall short in evaluating the diagnostic capabilities of LLMs in complex clinical cases. To address this gap, we have developed GynMedEval, a comprehensive dataset designed to assess the performance of LLMs in gynecologic disease diagnosis. This dataset is sourced from real‐world cases and the Chinese Clinical Case Outcomes Database, comprising 515 samples reviewed by physicians with senior titles (Associate Chief Physician or above). Each sample includes over 50 pieces of patient physiological characteristics and laboratory test data. We transformed the samples into a multiple‐choice format for evaluation. Several state‐of‐the‐art LLMs were assessed on the dataset under various diagnostic scenarios, including zero‐shot and few‐shot settings. The results revealed significant strengths in diagnosing common conditions, but none of the models achieved an accuracy rate above 90%. The establishment of the GynMedEval dataset addresses a critical gap in the evaluation of LLMs for gynecologic diagnosis. It will enable a deeper analysis of these models' performance, fostering their application in healthcare to enhance diagnostic accuracy, improve patient privacy, and ensure greater convenience.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

GynecologyLanguageGenital Diseases, FemaleHumansFemaleDatabases, FactualChinaDatasets as TopicLarge Language Models

Stichwörter der Autor:innen

clinical evaluationdatasetgynecologic diagnosislarge language model

Autor:innen und Institutionen

  • School of Communications and Information Engineering, North University of China, Taiyuan, China.
  • Laboratory of AI, Hangzhou Institute of Technology of Xidian University, Xidian University, Hangzhou, China.
  • Department of Obstetrics and Gynecology, Xingping People's Hospital, Xianyang, China.

Zitieren

APA

Dong, F., Zhang, H., Ren, Y., Li, W., Li, Y., Cui, G., & Liu, P. (2026). GynMedEval : A comprehensive dataset for evaluating the diagnostic capability of large language models in gynecology. International Journal of Gynecology & Obstetrics. https://doi.org/10.1002/ijgo.70437

Vancouver

Dong F, Zhang H, Ren Y, Li W, Li Y, Cui G, et al. GynMedEval : A comprehensive dataset for evaluating the diagnostic capability of large language models in gynecology. International Journal of Gynecology & Obstetrics. 2026. doi:10.1002/ijgo.70437

BibTeX
@article{Dong2026,
  author = {Dong, Fangchen and Zhang, HanZhi and Ren, Yi and Li, Weibin and Li, Yuan and Cui, Guangyu and Liu, Peixuan},
  title = {GynMedEval : A comprehensive dataset for evaluating the diagnostic capability of large language models in gynecology},
  journal = {International Journal of Gynecology & Obstetrics},
  year = {2026},
  doi = {10.1002/ijgo.70437}
}
Kostenlos starten →