EUVIMEDEUVIMED entdecken
Vergleichsstudie

Artificial intelligence meets HIV education: Comparing three large language models on accuracy, readability, and reliability

Özge Eren Korkmaz, Burcu Açıkalın Arıkan iD, Selda Sayın Kutlu, Figen Kaptan Aydoğmuş, Nurbanu Sezak

International Journal of STD & AIDS · 2026

DOI: 10.1177/09564624251372369 ↗

Kurz gefasst

Vergleichsstudie in der Zeitschrift International Journal of STD & AIDS, 2026. Verfasst von Özge Eren Korkmaz und 4 weiteren Autor:innen. Beteiligte Einrichtung: Epidemiology, Izmir Dokuz Eylul University, Izmir, Turkey. Hauptthemen laut MEDLINE: HIV Infections, Comprehension, Patient Education as Topic, Artificial Intelligence. Die Arbeit wurde bisher 6-mal zitiert (Crossref) und nennt 39 Literaturangaben.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

This study compares three large language models (LLMs) in answering common HIV questions, given ongoing concerns about their accuracy and reliability in patient education.

Methods

Models answered 63 HIV questions. Accuracy (5-point Likert), readability (Flesch-Kincaid, Gunning Fog, Coleman-Liau), and reliability (DISCERN, EQIP) were assessed.

Results

Claude 3.7 Sonnet showed significantly higher accuracy (4.54 ± 0.44) compared to ChatGPT-4o (4.29 ± 0.49) and Gemini Advanced 2.0 Flash (4.31 ± 0.50) ( p < .001). ChatGPT-4o had lower accuracy in disease definition, follow-up, and transmission routes, while Gemini Advanced 2.0 Flash performed poorly in daily life and treatment-related questions. Readability analyses indicated ChatGPT-4o produced the most accessible content according to Flesch-Kincaid and Coleman-Liau indices, whereas Claude 3.7 Sonnet was most comprehensible by Gunning Fog standards. Gemini Advanced 2.0 Flash consistently generated more complex texts across all readability measures ( p < .001). Regarding reliability, Claude 3.7 Sonnet achieved “good” quality on DISCERN, while others were rated “moderate” ( p = .059). On EQIP, Claude 3.7 Sonnet (median 61.8) and ChatGPT-4o (55.3) were classified as “good quality with minor limitations,” whereas Gemini Advanced 2.0 Flash (41.2) was rated “low quality” ( p = .049).

Conclusions

Claude 3.7 Sonnet is preferable for accuracy and reliability, while ChatGPT-4o offers superior readability. Selecting LLMs for HIV education should consider accuracy, readability, and reliability, emphasizing regular assessment of content quality and cultural sensitivity.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

HIV InfectionsComprehensionPatient Education as TopicArtificial IntelligenceLanguageHumansReproducibility of ResultsMaleFemaleSurveys and QuestionnairesAdultLarge Language Models

Stichwörter der Autor:innen

AIDSHuman immunodeficiency virusartificial intelligencepatient education as topic

Autor:innen und Institutionen

  • Epidemiology, Izmir Dokuz Eylul University, Izmir, Turkey.
  • Infectious Diseases and Clinical Microbiology, Izmir Democracy University Buca Seyfi Demirsoy Training Research Hospital, Izmir, Turkey.
  • Infectious Diseases and Clinical Microbiology, Denizli Pamukkale University, Denizli, Turkey.
  • Infectious Diseases and Clinical Microbiology, Izmir Katip Celebi University Atatürk Training and Research Hospital, Izmir, Turkey.
  • Infectious Diseases and Clinical Microbiology, Izmir Democracy University, Izmir, Turkey.

ORCID-Profile: Burcu Açıkalın Arıkan

Zitieren

APA

Eren Korkmaz, Ö., Açıkalın Arıkan, B., Sayın Kutlu, S., Kaptan Aydoğmuş, F., & Sezak, N. (2026). Artificial intelligence meets HIV education: Comparing three large language models on accuracy, readability, and reliability. International Journal of STD & AIDS. https://doi.org/10.1177/09564624251372369

Vancouver

Eren Korkmaz Ö, Açıkalın Arıkan B, Sayın Kutlu S, Kaptan Aydoğmuş F, Sezak N. Artificial intelligence meets HIV education: Comparing three large language models on accuracy, readability, and reliability. International Journal of STD & AIDS. 2026. doi:10.1177/09564624251372369

BibTeX
@article{ErenKorkmaz2026,
  author = {Eren Korkmaz, Özge and Açıkalın Arıkan, Burcu and Sayın Kutlu, Selda and Kaptan Aydoğmuş, Figen and Sezak, Nurbanu},
  title = {Artificial intelligence meets HIV education: Comparing three large language models on accuracy, readability, and reliability},
  journal = {International Journal of STD & AIDS},
  year = {2026},
  doi = {10.1177/09564624251372369}
}
Kostenlos starten →