Evaluating the quality of ChatGPT-generated medical information on major ophthalmic conditions: A comparative assessment against the EQIP tool and guidelines
PLOS One · 16.10.2025 · 20 (10) · e0334250
DOI: 10.1371/journal.pone.0334250 ↗Kurz gefasst
Vergleichsstudie in der Zeitschrift PLOS One, online erschienen am 16.10.2025. Verfasst von Mingfang Hu und 3 weiteren Autor:innen. Beteiligte Einrichtung: Ophthalmology Department, The First Affiliated Hospital of Zhengzhou University, Zhengzhou, China. Hauptthemen laut MEDLINE: Artificial Intelligence, Eye Diseases, Ophthalmology. Die Arbeit wurde bisher 5-mal zitiert (Crossref) und nennt 32 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: First Affiliated Hospital of Zhengzhou University.
Abstract (Originalsprache)
Background
The use of artificial intelligence for creating medical information is on the rise. Nonetheless, the accuracy and reliability of such information require thorough assessment. As a language model capable of generating text, ChatGPT needs a detailed examination of its effectiveness in the healthcare domain.
Objective
This research sought to evaluate the precision of medical data produced by ChatGPT-4o ( https://chat.openai.com/chat , accessed Mar. 12, 2025), concentrating on its capability to handle the top five ophthalmic issues that pose the greatest global health challenges. Furthermore, the investigation compared the AI’s answers to recognized medical guides.
Methods
This research employed an adapted version of the Ensuring Quality of Information for Patients (EQIP) instrument to evaluate the quality of ChatGPT’s replies. The guidelines for the five conditions were rephrased into pertinent queries. These queries were then fed into ChatGPT, employing benchmarking against established ophthalmology clinical guidelines, and the resulting answers were independently scrutinized for precision and consistency by two investigators. The consistency among raters was evaluated using Cohen’s kappa value.
Results
The median EQIP score across the five conditions was 18 (IQR 18-19). The modified EQIP instrument revealed a robust consensus between the two evaluators when assessing ChatGPT’s responses, as indicated by a Cohen’s kappa value of 0.926 (95% CI 0.875-0.977, P
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- Ophthalmology Department, The First Affiliated Hospital of Zhengzhou University, Zhengzhou, China.
ORCID-Profile: Mingfang Hu, Yuying Wang
Förderung
- First Affiliated Hospital of Zhengzhou University
Zitieren
APA
Hu, M., Zou, P., Li, T., & Wang, Y. (2025). Evaluating the quality of ChatGPT-generated medical information on major ophthalmic conditions: A comparative assessment against the EQIP tool and guidelines. PLOS One, 20(10), e0334250. https://doi.org/10.1371/journal.pone.0334250
Vancouver
Hu M, Zou P, Li T, Wang Y. Evaluating the quality of ChatGPT-generated medical information on major ophthalmic conditions: A comparative assessment against the EQIP tool and guidelines. PLOS One. 2025;20(10):e0334250. doi:10.1371/journal.pone.0334250
BibTeX
@article{Hu2025,
author = {Hu, Mingfang and Zou, Pingping and Li, Teng and Wang, Yuying},
title = {Evaluating the quality of ChatGPT-generated medical information on major ophthalmic conditions: A comparative assessment against the EQIP tool and guidelines},
journal = {PLOS One},
year = {2025},
volume = {20},
number = {10},
pages = {e0334250},
doi = {10.1371/journal.pone.0334250}
}
EUVIMED