EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC-SA 4.0

Assessing the response consistency in Chat Generative Pretrained Transformer: The case of “What is Ayurveda?”

Janmejaya Samal

International Journal of Health & Allied Sciences · 2025-01 · 14 (1) · 77-82

DOI: 10.4103/ijhas.ijhas_64_25 ↗

Kurz gefasst

Fachartikel in der Zeitschrift International Journal of Health & Allied Sciences, online erschienen am 01.10.2025. Verfasst von Janmejaya Samal. Die Arbeit nennt 39 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-SA 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

INTRODUCTION: Chat Generative Pretrained Transformer (ChatGPT) is a language-based chatbot launched in November 2022. This study was carried out to understand the consistency of the responses of ChatGPT to a single question when asked 10 different times, consecutively, within a particular time frame. MATERIALS AND METHODS: The study followed two steps: in the first step, only one question, “What is Ayurveda ?” was asked, followed by a conversation for bibliographic citations. The responses obtained were analyzed statistically, and the key concepts from the responses were free listed. Statistical analysis was conducted using Jamovi (version 2.3.28). RESULTS: The median word count was 334 (standard deviation [SD] = 32.87, interquartile range [IQR] =313–354), sentences were 18 (SD = 2.36, IQR = 17.0–20.3), headings were 1.50 (SD = 0.82, IQR = 1–2), and the subheadings were 8 (SD = 2.08, IQR = 7–9). The linear regression between the number of words and other particulars showed a statistically significant association with the number of sentences ( P = 0.003, 95% confidence interval). The free listing of the key concepts was done under three headings: core principles, treatment practices, and modern relevance. Four concepts were found to have the highest frequencies in the ChatGPT responses: Doshas ( n = 10, 100%), Panchakarma ( n = 7, 70%), Pancha Mahabhutas ( n = 7, 70%), and herbal medicine ( n = 6, 60%). In the conversation, ChatGPT provided 21 different bibliographic citations with three rounds of questions; however, none of the citations were found to exist. CONCLUSIONS: ChatGPT can be useful in many ways in education and research. However, it should be used with a pinch of salt as ChatGPT itself says, “ChatGPT can make mistakes. Check important info.”

Zitieren

APA

Samal, J. (2025). Assessing the response consistency in Chat Generative Pretrained Transformer: The case of “What is Ayurveda?”. International Journal of Health & Allied Sciences, 14(1), 77-82. https://doi.org/10.4103/ijhas.ijhas_64_25

Vancouver

Samal J. Assessing the response consistency in Chat Generative Pretrained Transformer: The case of “What is Ayurveda?”. International Journal of Health & Allied Sciences. 2025;14(1):77-82. doi:10.4103/ijhas.ijhas_64_25

BibTeX
@article{Samal2025,
  author = {Samal, Janmejaya},
  title = {Assessing the response consistency in Chat Generative Pretrained Transformer: The case of “What is Ayurveda?”},
  journal = {International Journal of Health & Allied Sciences},
  year = {2025},
  volume = {14},
  number = {1},
  pages = {77--82},
  doi = {10.4103/ijhas.ijhas_64_25}
}
Kostenlos starten →