EUVIMEDEUVIMED entdecken
Open Access

Assessing the Efficacy of ChatGPT Versus Human Researchers in Identifying Relevant Studies on mHealth Interventions for Improving Medication Adherence in Patients With Ischemic Stroke When Conducting Systematic Reviews: Comparative Analysis

Suebsarn Ruksakulpiwat iD, Lalipat Phianhasin iD, Chitchanok Benjasirisan iD, Kedong Ding iD, Anuoluwapo Ajibade iD, Ayanesh Kumar iD, Cassie Stewart iD

JMIR mHealth and uHealth · 06.05.2024 · 12 · e51526

DOI: 10.2196/51526 ↗

Kurz gefasst

Fachartikel in der Zeitschrift JMIR mHealth and uHealth, online erschienen am 06.05.2024. Verfasst von Suebsarn Ruksakulpiwat und 6 weiteren Autor:innen. Beteiligte Einrichtung: Department of Medical Nursing, Faculty of Nursing, Mahidol University, Bangkok, Thailand. Hauptthemen laut MEDLINE: Medication Adherence, Telemedicine, Drug Monitoring. Die Arbeit wurde bisher 19-mal zitiert (Crossref) und nennt 31 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

ChatGPT by OpenAI emerged as a potential tool for researchers, aiding in various aspects of research. One such application was the identification of relevant studies in systematic reviews. However, a comprehensive comparison of the efficacy of relevant study identification between human researchers and ChatGPT has not been conducted.

Objective

This study aims to compare the efficacy of ChatGPT and human researchers in identifying relevant studies on medication adherence improvement using mobile health interventions in patients with ischemic stroke during systematic reviews.

Methods

This study used the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. Four electronic databases, including CINAHL Plus with Full Text, Web of Science, PubMed, and MEDLINE, were searched to identify articles published from inception until 2023 using search terms based on MeSH (Medical Subject Headings) terms generated by human researchers versus ChatGPT. The authors independently screened the titles, abstracts, and full text of the studies identified through separate searches conducted by human researchers and ChatGPT. The comparison encompassed several aspects, including the ability to retrieve relevant studies, accuracy, efficiency, limitations, and challenges associated with each method.

Results

A total of 6 articles identified through search terms generated by human researchers were included in the final analysis, of which 4 (67%) reported improvements in medication adherence after the intervention. However, 33% (2/6) of the included studies did not clearly state whether medication adherence improved after the intervention. A total of 10 studies were included based on search terms generated by ChatGPT, of which 6 (60%) overlapped with studies identified by human researchers. Regarding the impact of mobile health interventions on medication adherence, most included studies (8/10, 80%) based on search terms generated by ChatGPT reported improvements in medication adherence after the intervention. However, 20% (2/10) of the studies did not clearly state whether medication adherence improved after the intervention. The precision in accurately identifying relevant studies was higher in human researchers (0.86) than in ChatGPT (0.77). This is consistent with the percentage of relevance, where human researchers (9.8%) demonstrated a higher percentage of relevance than ChatGPT (3%). However, when considering the time required for both humans and ChatGPT to identify relevant studies, ChatGPT substantially outperformed human researchers as it took less time to identify relevant studies.

Conclusions

Our comparative analysis highlighted the strengths and limitations of both approaches. Ultimately, the choice between human researchers and ChatGPT depends on the specific requirements and objectives of each review, but the collaborative synergy of both approaches holds the potential to advance evidence-based research and decision-making in the health care field.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Medication AdherenceTelemedicineDrug MonitoringHumansIschemic StrokeSystematic Reviews as TopicResearch Personnel

Stichwörter der Autor:innen

ChatGPTischemic strokemHealthmedication adherencemobile healthmobile phonesystematic reviews

Autor:innen und Institutionen

  • Department of Medical Nursing, Faculty of Nursing, Mahidol University, Bangkok, Thailand.
  • Jack, Joseph and Morton Mandel School of Applied Social Sciences, Case Western Reserve University, Cleveland, OH, United States.
  • College of Art and Science, Department of Anthropology, Case Western Reserve University, Cleveland, OH, United States.
  • School of Medicine, Case Western Reserve University, Cleveland, OH, United States.
  • Frances Payne Bolton School of Nursing, Case Western Reserve University, Cleveland, OH, United States.

ORCID-Profile: Suebsarn Ruksakulpiwat, Lalipat Phianhasin, Chitchanok Benjasirisan, Kedong Ding, Anuoluwapo Ajibade, Ayanesh Kumar, Cassie Stewart

Zitieren

APA

Ruksakulpiwat, S., Phianhasin, L., Benjasirisan, C., Ding, K., Ajibade, A., Kumar, A., & Stewart, C. (2024). Assessing the Efficacy of ChatGPT Versus Human Researchers in Identifying Relevant Studies on mHealth Interventions for Improving Medication Adherence in Patients With Ischemic Stroke When Conducting Systematic Reviews: Comparative Analysis. JMIR mHealth and uHealth, 12, e51526. https://doi.org/10.2196/51526

Vancouver

Ruksakulpiwat S, Phianhasin L, Benjasirisan C, Ding K, Ajibade A, Kumar A, et al. Assessing the Efficacy of ChatGPT Versus Human Researchers in Identifying Relevant Studies on mHealth Interventions for Improving Medication Adherence in Patients With Ischemic Stroke When Conducting Systematic Reviews: Comparative Analysis. JMIR mHealth and uHealth. 2024;12:e51526. doi:10.2196/51526

BibTeX
@article{Ruksakulpiwat2024,
  author = {Ruksakulpiwat, Suebsarn and Phianhasin, Lalipat and Benjasirisan, Chitchanok and Ding, Kedong and Ajibade, Anuoluwapo and Kumar, Ayanesh and Stewart, Cassie},
  title = {Assessing the Efficacy of ChatGPT Versus Human Researchers in Identifying Relevant Studies on mHealth Interventions for Improving Medication Adherence in Patients With Ischemic Stroke When Conducting Systematic Reviews: Comparative Analysis},
  journal = {JMIR mHealth and uHealth},
  year = {2024},
  volume = {12},
  pages = {e51526},
  doi = {10.2196/51526}
}
Kostenlos starten →