LLMs and Stargardt's Disease
Journal of Surgery Care · 03.11.2025 · 04 (04) · 01-08
DOI: 10.33140/jsc.04.04.03 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Surgery Care, online erschienen am 03.11.2025. Verfasst von Gloria Wu und 7 weiteren Autor:innen.
Abstract (Originalsprache)
Background/Objective: Stargardt's disease is the most common form of inherited juvenile macular degeneration affecting 1 in 8,000-10,000 individuals worldwide, with a slight predominance towards females. As large language models (LLMs) increasingly serve as sources of health information, understanding their effectiveness in providing accurate information about rare genetic conditions becomes essential. This study aims to evaluate and compare four major LLMs (ChatGPT, Gemini, Claude, and Character.ai) regarding Stargardt's disease information delivery across different genders.
Methods
Four LLMs were queried using standardized prompts simulating a 14-year-old patient (male/female) newly diagnosed with Stargardt's disease. Responses were analyzed for word count, readability (Flesch-Kincaid Grade Level), response time, and content similarity using cosine analysis.
Results
Significant variations existed across LLMs. Word counts ranged from 53 to 769 words, with Gemini producing the most comprehensive responses (female: 769 words, male: 708 words) and Character.ai the most concise (female: 74 words, male: 53 words). Flesch-Kincaid scores indicated a readability level suitable for high school to college (5.4- 10.8). Response times varied from 5.5 to 13.8 seconds. Cosine similarity scores showed moderate concordance (58.5- 78.3%) between model pairs. All LLMs recommended physician consultation and genetic testing, but varied significantly in the provision of emotional support and comprehensive information.
Conclusions
While all LLMs provided appropriate referral recommendations, substantial disparities exist in the depth of content, readability, and information delivery. No LLM consistently addressed the full spectrum of Stargardt's disease management, including specialist referrals, genetic counseling, and available therapies. These findings underscore the importance of physician oversight and standardization in AI-generated healthcare information to ensure the accuracy of care delivery
Zitieren
APA
Wu, G., Bojja, L., Sidhu, A., Khan, O., Paliath-Pathiyal, H., Liu, B., Wang, M. C., & Chim, I. (2025). LLMs and Stargardt's Disease. Journal of Surgery Care, 04(04), 01-08. https://doi.org/10.33140/jsc.04.04.03
Vancouver
Wu G, Bojja L, Sidhu A, Khan O, Paliath-Pathiyal H, Liu B, et al. LLMs and Stargardt's Disease. Journal of Surgery Care. 2025;04(04):01-08. doi:10.33140/jsc.04.04.03
BibTeX
@article{Wu2025,
author = {Wu, Gloria and Bojja, Lochan and Sidhu, Aadjot and Khan, Obaid and Paliath-Pathiyal, Hrishi and Liu, Bojia and Wang, Margaret C and Chim, Ivan},
title = {LLMs and Stargardt's Disease},
journal = {Journal of Surgery Care},
year = {2025},
volume = {04},
number = {04},
pages = {01--08},
doi = {10.33140/jsc.04.04.03}
}
EUVIMED