Use of large language models to identify pseudo‐information: Implications for health information
Health Information & Libraries Journal · 2025-12 · 42 (4) · 568-577
DOI: 10.1111/hir.12569 ↗Kurz gefasst
Fachartikel in der Zeitschrift Health Information & Libraries Journal, online erschienen am 20.03.2025. Verfasst von Boris Schmitz. Beteiligte Einrichtung: Department of Rehabilitation Sciences, Faculty of Health, University of Witten/Herdecke, Witten, Germany. Hauptthemen laut MEDLINE: Artificial Intelligence, Language. Die Arbeit wurde bisher 4-mal zitiert (Crossref) und nennt 17 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: European Commission within the Horizon 2020 Framework Program.
Abstract (Originalsprache)
Background
Open‐access scientific research is an essential source of health‐related information and self‐education. Artificial intelligence‐based large language models (LMMs) may be used to identify erroneous health information.
Objective
To investigate to what extent LMMs can be used to identify pseudo‐information.
Methods
Four common LMM applications (ChatGPT‐4o, Claude 3.5 Sonnet, Gemini and Copilot) were used to investigate their capability to indicate erroneous information provided in an open‐access article.
Results
Initially, ChatGPT‐4o and Claude were able to mark the provided article as an unreliable information source, identifying most of the inaccuracy problems. The assessments provided by Gemini and Copilot were inaccurate, as several critical aspects were not identified or were misinterpreted. During the validation phase, the initially accurate assessment of ChatGPT‐4o was not reproducible, and only Claude was able to detect several critical issues in this phase. The verdicts of Copilot and Gemini remained largely unaltered.
Discussion
Large heterogeneity exists between LMMs in identifying inaccurate pseudo‐information. Replication in LMM output may constitute a significant hurdle in their application.
Conclusion
The accuracy of LMMs needs to be further improved until they can be reliably used by patients for health‐related online information and as assistant tools for health information and library services workers without restriction.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Rehabilitation Sciences, Faculty of Health, University of Witten/Herdecke, Witten, Germany.
- DRV Clinic Königsfeld, Center for Medical Rehabilitation, Ennepetal, Germany.
ORCID-Profile: Boris Schmitz
Förderung
- European Commission within the Horizon 2020 Framework Program
Zitieren
APA
Schmitz, B. (2025). Use of large language models to identify pseudo‐information: Implications for health information. Health Information & Libraries Journal, 42(4), 568-577. https://doi.org/10.1111/hir.12569
Vancouver
Schmitz B. Use of large language models to identify pseudo‐information: Implications for health information. Health Information & Libraries Journal. 2025;42(4):568-577. doi:10.1111/hir.12569
BibTeX
@article{Schmitz2025,
author = {Schmitz, Boris},
title = {Use of large language models to identify pseudo‐information: Implications for health information},
journal = {Health Information & Libraries Journal},
year = {2025},
volume = {42},
number = {4},
pages = {568--577},
doi = {10.1111/hir.12569}
}
EUVIMED