EUVIMEDEUVIMED entdecken
VergleichsstudieLeitlinieOpen Access · CC BY 4.0

Evaluating the quality of ChatGPT-generated medical information on major ophthalmic conditions: A comparative assessment against the EQIP tool and guidelines

Mingfang Hu iD, Pingping Zou, Teng Li, Yuying Wang iD

PLOS One · 16.10.2025 · 20 (10) · e0334250

DOI: 10.1371/journal.pone.0334250 ↗

Kurz gefasst

Vergleichsstudie in der Zeitschrift PLOS One, online erschienen am 16.10.2025. Verfasst von Mingfang Hu und 3 weiteren Autor:innen. Beteiligte Einrichtung: Ophthalmology Department, The First Affiliated Hospital of Zhengzhou University, Zhengzhou, China. Hauptthemen laut MEDLINE: Artificial Intelligence, Eye Diseases, Ophthalmology. Die Arbeit wurde bisher 5-mal zitiert (Crossref) und nennt 32 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: First Affiliated Hospital of Zhengzhou University.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

The use of artificial intelligence for creating medical information is on the rise. Nonetheless, the accuracy and reliability of such information require thorough assessment. As a language model capable of generating text, ChatGPT needs a detailed examination of its effectiveness in the healthcare domain.

Objective

This research sought to evaluate the precision of medical data produced by ChatGPT-4o ( https://chat.openai.com/chat , accessed Mar. 12, 2025), concentrating on its capability to handle the top five ophthalmic issues that pose the greatest global health challenges. Furthermore, the investigation compared the AI’s answers to recognized medical guides.

Methods

This research employed an adapted version of the Ensuring Quality of Information for Patients (EQIP) instrument to evaluate the quality of ChatGPT’s replies. The guidelines for the five conditions were rephrased into pertinent queries. These queries were then fed into ChatGPT, employing benchmarking against established ophthalmology clinical guidelines, and the resulting answers were independently scrutinized for precision and consistency by two investigators. The consistency among raters was evaluated using Cohen’s kappa value.

Results

The median EQIP score across the five conditions was 18 (IQR 18-19). The modified EQIP instrument revealed a robust consensus between the two evaluators when assessing ChatGPT’s responses, as indicated by a Cohen’s kappa value of 0.926 (95% CI 0.875-0.977, P

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Artificial IntelligenceEye DiseasesOphthalmologyHumansPractice Guidelines as TopicReproducibility of ResultsGenerative Artificial Intelligence

Autor:innen und Institutionen

  • Ophthalmology Department, The First Affiliated Hospital of Zhengzhou University, Zhengzhou, China.

ORCID-Profile: Mingfang Hu, Yuying Wang

Förderung

  • First Affiliated Hospital of Zhengzhou University

Zitieren

APA

Hu, M., Zou, P., Li, T., & Wang, Y. (2025). Evaluating the quality of ChatGPT-generated medical information on major ophthalmic conditions: A comparative assessment against the EQIP tool and guidelines. PLOS One, 20(10), e0334250. https://doi.org/10.1371/journal.pone.0334250

Vancouver

Hu M, Zou P, Li T, Wang Y. Evaluating the quality of ChatGPT-generated medical information on major ophthalmic conditions: A comparative assessment against the EQIP tool and guidelines. PLOS One. 2025;20(10):e0334250. doi:10.1371/journal.pone.0334250

BibTeX
@article{Hu2025,
  author = {Hu, Mingfang and Zou, Pingping and Li, Teng and Wang, Yuying},
  title = {Evaluating the quality of ChatGPT-generated medical information on major ophthalmic conditions: A comparative assessment against the EQIP tool and guidelines},
  journal = {PLOS One},
  year = {2025},
  volume = {20},
  number = {10},
  pages = {e0334250},
  doi = {10.1371/journal.pone.0334250}
}
Kostenlos starten →