EUVIMEDEUVIMED entdecken
Crossref

An Empirical Study Evaluating ChatGPT 's Performance in Generating Search Strategies for Systematic Reviews

Fei Yu, Heather Kincaide, Rebecca Beth Carlson

Proceedings of the Association for Information Science and Technology · 2024-10 · 61 (1) · 423-434

DOI: 10.1002/pra2.1039 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the Association for Information Science and Technology, online erschienen am 15.10.2024. Verfasst von Fei Yu und 2 weiteren Autor:innen. Die Arbeit wurde bisher 7-mal zitiert (Crossref) und nennt 28 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

This study evaluated the performance of ChatGPT‐3.5 and ChatGPT‐4 in developing search strategies for systematic reviews. Using the Peer Review of Electronic Search Strategies (PRESS) framework, we employed a two‐round testing format for each version. In the first round, both versions displayed comparable competencies when assessed quantitatively by the PRESS measures. However, qualitative feedback from two professional health sciences librarians indicated that ChatGPT‐4 outperformed ChatGPT‐3.5, particularly in suggesting MeSH term inclusion and refining search strategy formulations. In the second round, prompts were refined based on the feedback from the previous round of testing. Both qualitative and quantitative evaluation results confirmed ChatGPT‐4's superiority. This study provides empirical evidence of advancements in language model capabilities, highlighting ChatGPT‐4's enhanced efficiency and accuracy in developing search strategies for systematic reviews.

Zitieren

APA

Yu, F., Kincaide, H., & Carlson, R. B. (2024). An Empirical Study Evaluating ChatGPT 's Performance in Generating Search Strategies for Systematic Reviews. Proceedings of the Association for Information Science and Technology, 61(1), 423-434. https://doi.org/10.1002/pra2.1039

Vancouver

Yu F, Kincaide H, Carlson RB. An Empirical Study Evaluating ChatGPT 's Performance in Generating Search Strategies for Systematic Reviews. Proceedings of the Association for Information Science and Technology. 2024;61(1):423-434. doi:10.1002/pra2.1039

BibTeX
@article{Yu2024,
  author = {Yu, Fei and Kincaide, Heather and Carlson, Rebecca Beth},
  title = {An Empirical Study Evaluating ChatGPT 's Performance in Generating Search Strategies for Systematic Reviews},
  journal = {Proceedings of the Association for Information Science and Technology},
  year = {2024},
  volume = {61},
  number = {1},
  pages = {423--434},
  doi = {10.1002/pra2.1039}
}
Kostenlos starten →