EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Use of large language models to identify pseudo‐information: Implications for health information

Boris Schmitz iD

Health Information & Libraries Journal · 2025-12 · 42 (4) · 568-577

DOI: 10.1111/hir.12569 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Health Information & Libraries Journal, online erschienen am 20.03.2025. Verfasst von Boris Schmitz. Beteiligte Einrichtung: Department of Rehabilitation Sciences, Faculty of Health, University of Witten/Herdecke, Witten, Germany. Hauptthemen laut MEDLINE: Artificial Intelligence, Language. Die Arbeit wurde bisher 4-mal zitiert (Crossref) und nennt 17 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: European Commission within the Horizon 2020 Framework Program.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Open‐access scientific research is an essential source of health‐related information and self‐education. Artificial intelligence‐based large language models (LMMs) may be used to identify erroneous health information.

Objective

To investigate to what extent LMMs can be used to identify pseudo‐information.

Methods

Four common LMM applications (ChatGPT‐4o, Claude 3.5 Sonnet, Gemini and Copilot) were used to investigate their capability to indicate erroneous information provided in an open‐access article.

Results

Initially, ChatGPT‐4o and Claude were able to mark the provided article as an unreliable information source, identifying most of the inaccuracy problems. The assessments provided by Gemini and Copilot were inaccurate, as several critical aspects were not identified or were misinterpreted. During the validation phase, the initially accurate assessment of ChatGPT‐4o was not reproducible, and only Claude was able to detect several critical issues in this phase. The verdicts of Copilot and Gemini remained largely unaltered.

Discussion

Large heterogeneity exists between LMMs in identifying inaccurate pseudo‐information. Replication in LMM output may constitute a significant hurdle in their application.

Conclusion

The accuracy of LMMs needs to be further improved until they can be reliably used by patients for health‐related online information and as assistant tools for health information and library services workers without restriction.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Artificial IntelligenceLanguageHumansLarge Language Models

Stichwörter der Autor:innen

Artificial Intelligence (AI)consumer health informationinternetmisinformationpatient information

Autor:innen und Institutionen

  • Department of Rehabilitation Sciences, Faculty of Health, University of Witten/Herdecke, Witten, Germany.
  • DRV Clinic Königsfeld, Center for Medical Rehabilitation, Ennepetal, Germany.

ORCID-Profile: Boris Schmitz

Förderung

  • European Commission within the Horizon 2020 Framework Program

Zitieren

APA

Schmitz, B. (2025). Use of large language models to identify pseudo‐information: Implications for health information. Health Information & Libraries Journal, 42(4), 568-577. https://doi.org/10.1111/hir.12569

Vancouver

Schmitz B. Use of large language models to identify pseudo‐information: Implications for health information. Health Information & Libraries Journal. 2025;42(4):568-577. doi:10.1111/hir.12569

BibTeX
@article{Schmitz2025,
  author = {Schmitz, Boris},
  title = {Use of large language models to identify pseudo‐information: Implications for health information},
  journal = {Health Information & Libraries Journal},
  year = {2025},
  volume = {42},
  number = {4},
  pages = {568--577},
  doi = {10.1111/hir.12569}
}
Kostenlos starten →