Evaluating the accuracy, reliability, and readability of AI chatbots in delivering postpartum depression information
Frontiers in Psychiatry · 12.08.2026 · 17
DOI: 10.3389/fpsyt.2026.1881536 ↗Kurz gefasst
Fachartikel in der Zeitschrift Frontiers in Psychiatry, online erschienen am 12.08.2026. Verfasst von Man Yang und 8 weiteren Autor:innen. Beteiligte Einrichtung: Department of Obstetrics, Shenzhen Nanshan Maternity and Child Healthcare Hospital, Shenzhen, China. Die Arbeit nennt 29 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Themen: Depression
Abstract (Originalsprache)
Background
Postpartum depression (PPD) is a common perinatal psychiatric disorder with significant implications for maternal and infant health. Artificial intelligence (AI) chatbots have emerged as widely accessible tools for health information, but their performance in providing accurate, reliable, and readable PPD related information remains underexplored.
Methods
We evaluated six AI chatbots, ChatGPT-5, ChatGPT-4o, Claude Sonnet 4.5, DeepSeek-V3.2, DeepSeek-R1, and Gemini 2.5 Pro, using 200 standardized multiple choice questions (MCQs) on PPD to assess validity. ChatGPT-4o and DeepSeek-R1 were included only in the MCQ based validity analysis as earlier version comparators. Reliability and readability were further assessed using 20 core public education questions in the four latest models: ChatGPT-5, Claude Sonnet 4.5, DeepSeek-V3.2, and Gemini 2.5 Pro. Chatbot performance was assessed across three dimensions: validity, reliability, and readability. Each MCQ was presented three times independently, and each core public education question was assessed once per model.
Results
In the six model MCQ based validity analysis, ChatGPT-5 achieved the highest overall accuracy on MCQs (97.50% ± 0.50%). In the four models reliability and readability analyses, ChatGPT-5 obtained the highest DISCERN, EQIP, and GQS scores, suggesting relatively better content quality and user oriented usefulness. However, JAMA benchmark scores were low across all models, including ChatGPT-5, indicating limited transparency, source attribution, currency, and disclosure. All models produced outputs exceeding the recommended sixth grade reading level, although ChatGPT-5 and Gemini 2.5 Pro were relatively more accessible.
Conclusion
AI chatbots, particularly ChatGPT-5, showed potential as supplementary tools for providing postpartum depression related information, especially in standardized MCQ based assessment. However, this study did not evaluate clinical safety, patient comprehension, user behavior, or real world effectiveness, and suboptimal readability may limit accessibility for users with lower health or digital literacy. Inadequate transparency, limited source attribution, and suboptimal readability indicate that AI chatbots should not be used as autonomous sources of postpartum mental health guidance and should not replace professional assessment or care.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Obstetrics, Shenzhen Nanshan Maternity and Child Healthcare Hospital, Shenzhen, China.
Zitieren
APA
Yang, M., Liu, H., Lin, S., Liu, L., Wang, Y., Qiu, X., Wei, L., Zhou, L., & Wang, X. (2026). Evaluating the accuracy, reliability, and readability of AI chatbots in delivering postpartum depression information. Frontiers in Psychiatry, 17. https://doi.org/10.3389/fpsyt.2026.1881536
Vancouver
Yang M, Liu H, Lin S, Liu L, Wang Y, Qiu X, et al. Evaluating the accuracy, reliability, and readability of AI chatbots in delivering postpartum depression information. Frontiers in Psychiatry. 2026;17. doi:10.3389/fpsyt.2026.1881536
BibTeX
@article{Yang2026,
author = {Yang, Man and Liu, Hui and Lin, Shuyan and Liu, Linyan and Wang, Yunxia and Qiu, Xiuling and Wei, Ling and Zhou, Li and Wang, Xinxin},
title = {Evaluating the accuracy, reliability, and readability of AI chatbots in delivering postpartum depression information},
journal = {Frontiers in Psychiatry},
year = {2026},
volume = {17},
doi = {10.3389/fpsyt.2026.1881536}
}
EUVIMED