Human versus artificial intelligence‐generated arthroplasty literature: A single‐blinded analysis of perceived communication, quality, and authorship source
The International Journal of Medical Robotics and Computer Assisted Surgery · 2024
DOI: 10.1002/rcs.2621 ↗Kurz gefasst
Randomisierte kontrollierte Studie in der Zeitschrift The International Journal of Medical Robotics and Computer Assisted Surgery, 2024. Verfasst von Kyle W Lawrence und 5 weiteren Autor:innen. Beteiligte Einrichtung: Department of Orthopedic Surgery, NYU Langone Health, New York, New York, USA. Hauptthemen laut MEDLINE: Authorship, Artificial Intelligence. Die Arbeit wurde bisher 16-mal zitiert (Crossref) und nennt 24 Literaturangaben.
Abstract (Originalsprache)
Background
Large language models (LLM) have unknown implications for medical research. This study assessed whether LLM‐generated abstracts are distinguishable from human‐written abstracts and to compare their perceived quality.
Methods
The LLM ChatGPT was used to generate 20 arthroplasty abstracts (AI‐generated) based on full‐text manuscripts, which were compared to originally published abstracts (human‐written). Six blinded orthopaedic surgeons rated abstracts on overall quality, communication, and confidence in the authorship source. Authorship‐confidence scores were compared to a test value representing complete inability to discern authorship.
Results
Modestly increased confidence in human authorship was observed for human‐written abstracts compared with AI‐generated abstracts ( p = 0.028), though AI‐generated abstract authorship‐confidence scores were statistically consistent with inability to discern authorship ( p = 0.999). Overall abstract quality was higher for human‐written abstracts ( p = 0.019).
Conclusions
AI‐generated abstracts' absolute authorship‐confidence ratings demonstrated difficulty in discerning authorship but did not achieve the perceived quality of human‐written abstracts. Caution is warranted in implementing LLMs into scientific writing.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Orthopedic Surgery, NYU Langone Health, New York, New York, USA.
ORCID-Profile: Kyle W Lawrence, Akram A Habibi
Zitieren
APA
Lawrence, K. W., Habibi, A. A., Ward, S. A., Lajam, C. M., Schwarzkopf, R., & Rozell, J. C. (2024). Human versus artificial intelligence‐generated arthroplasty literature: A single‐blinded analysis of perceived communication, quality, and authorship source. The International Journal of Medical Robotics and Computer Assisted Surgery. https://doi.org/10.1002/rcs.2621
Vancouver
Lawrence KW, Habibi AA, Ward SA, Lajam CM, Schwarzkopf R, Rozell JC. Human versus artificial intelligence‐generated arthroplasty literature: A single‐blinded analysis of perceived communication, quality, and authorship source. The International Journal of Medical Robotics and Computer Assisted Surgery. 2024. doi:10.1002/rcs.2621
BibTeX
@article{Lawrence2024,
author = {Lawrence, Kyle W and Habibi, Akram A and Ward, Spencer A and Lajam, Claudette M and Schwarzkopf, Ran and Rozell, Joshua C},
title = {Human versus artificial intelligence‐generated arthroplasty literature: A single‐blinded analysis of perceived communication, quality, and authorship source},
journal = {The International Journal of Medical Robotics and Computer Assisted Surgery},
year = {2024},
doi = {10.1002/rcs.2621}
}
EUVIMED