EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC-ND 4.0

Reliability, Readability, and Safety of Generative AI Chatbots for Osteoporosis Health Counseling: A Multidimensional Cross‐Sectional Comparative Study

Huimin Xu iD, Liang Guo, Li Shen, Ting Zhu, Xiaona Huang, Liwen Ding iD

Health Science Reports · 2026-10 · 9 (10)

DOI: 10.1002/hsr2.73381 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Health Science Reports, online erschienen am 10.10.2026. Verfasst von Huimin Xu und 5 weiteren Autor:innen. Beteiligte Einrichtung: Yueyang Hospital of Integrated Traditional Chinese and Western Medicine, Shanghai University of Traditional Chinese Medicine Shanghai China. Die Arbeit nennt 34 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background and Aims Generative artificial intelligence chatbots are increasingly used for health information, but their reliability, readability, and safety in osteoporosis counseling remain uncertain. This study compared four widely available chatbots under standardized conditions.

Methods

Twenty‐five standardized osteoporosis questions were submitted once to ChatGPT‐5, Gemini 2.5 Pro, Copilot, and Perplexity Pro, generating 100 responses. Two model‐blinded specialists evaluated the final responses using DISCERN, EQIP, the Global Quality Scale, JAMA benchmarks, and the NOHARM framework. Six automated indices assessed readability. Because each question was answered by all four models, outcomes were compared using Friedman tests, followed by paired Wilcoxon signed‐rank tests with Holm adjustment.

Results

DISCERN scores differed across models ( p < 0.001; Kendall's W = 0.82, 95% CI 0.76–0.89). Median scores were 55.0 for Perplexity Pro, 54.0 for Copilot, 44.0 for Gemini 2.5 Pro, and 42.0 for ChatGPT‐5. JAMA benchmark scores did not differ significantly ( p = 0.63). All six readability indices differed across models (all p < 0.001). Perplexity Pro had the highest median Flesch–Kincaid Grade Level score at 16.18, whereas Copilot had the lowest at 9.49; all models exceeded the sixth‐grade benchmark. The qualitative safety assessment also identified clinically relevant omissions and an unsupported claim.

Conclusion

In this set of osteoporosis questions, retrieval‐enabled systems achieved higher scores on selected information‐quality measures but produced more difficult text. These findings are limited to the tested models, prompts, and evaluation period and do not establish a universal effect of retrieval augmentation.

Schlagwörter

Stichwörter der Autor:innen

clinical safetyhealth informationlarge language modelsosteoporosispatient educationreadability

Autor:innen und Institutionen

  • Yueyang Hospital of Integrated Traditional Chinese and Western Medicine, Shanghai University of Traditional Chinese Medicine Shanghai China.

ORCID-Profile: Huimin Xu, Liwen Ding

Zitieren

APA

Xu, H., Guo, L., Shen, L., Zhu, T., Huang, X., & Ding, L. (2026). Reliability, Readability, and Safety of Generative AI Chatbots for Osteoporosis Health Counseling: A Multidimensional Cross‐Sectional Comparative Study. Health Science Reports, 9(10). https://doi.org/10.1002/hsr2.73381

Vancouver

Xu H, Guo L, Shen L, Zhu T, Huang X, Ding L. Reliability, Readability, and Safety of Generative AI Chatbots for Osteoporosis Health Counseling: A Multidimensional Cross‐Sectional Comparative Study. Health Science Reports. 2026;9(10). doi:10.1002/hsr2.73381

BibTeX
@article{Xu2026,
  author = {Xu, Huimin and Guo, Liang and Shen, Li and Zhu, Ting and Huang, Xiaona and Ding, Liwen},
  title = {Reliability, Readability, and Safety of Generative AI Chatbots for Osteoporosis Health Counseling: A Multidimensional Cross‐Sectional Comparative Study},
  journal = {Health Science Reports},
  year = {2026},
  volume = {9},
  number = {10},
  doi = {10.1002/hsr2.73381}
}
Kostenlos starten →