The role of large language models in the writing of surgical reviews: fact or fantasy?
Langenbeck's Archives of Surgery · 03.07.2026 · 411 (1)
DOI: 10.1007/s00423-026-04095-2 ↗Kurz gefasst
Fachartikel in der Zeitschrift Langenbeck's Archives of Surgery, online erschienen am 03.07.2026. Verfasst von Daphne E DeTemple und 7 weiteren Autor:innen. Beteiligte Einrichtung: Department for General-, Visceral- and Transplant Surgery, Hannover Medical School, Hannover, Germany. ddetemple@ukaachen.de. Hauptthemen laut MEDLINE: Large Language Models, Writing, Artificial Intelligence. Die Arbeit nennt 34 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Universitätsklinikum RWTH Aachen.
Abstract (Originalsprache)
Background
Amidst the current enthusiasm concerning artificial intelligence and its possible application in the composition of different kinds of scientific and non-scientific written documents, we evaluated the usage of artificial intelligence for writing surgical short reviews.
Methods
In order to assess the formal and content quality of AI-generated texts compared to human written texts, ten AI-based text generators (five chatbots and five content creators) and four surgeons in training received the same prompt for a short scientific article on a liver surgery theme. All texts were anonymized and subsequently evaluated by three experienced liver surgeons based on a pre-defined scoring scheme, as well as for quality of references and readability according to readability indices. Furthermore, all texts were tested for plagiarism using PlagScan.
Results
Overall percentage of correct assessment for AI/non-AI generation by experienced surgeons lay at 78.57%. Human written text had a mean word count of 1054 versus 874 in AI-generated text, with a higher mean Flesh Reading Ease Score (FRE, 26.2 ± 5.1 versus 17.7 ± 6.1). References were PubMed-listed in 100% for human versus 46% for AI-generated text, with only one non-human text reaching 100% formally correct citation of references. PlagScan found 6.4%±1.3 mean resemblance to existing texts for human versus 7.6%±4.5 for AI-generated text.
Discussion
Overall, AI could already mislead experienced scientific surgeons in 26.7% of cases into believing it to be human. However, formal requirements, especially considering referencing, are still in great need of improvement with only one of AI-generated articles fulfilling our quality requirements.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department for General-, Visceral- and Transplant Surgery, Hannover Medical School, Hannover, Germany. ddetemple@ukaachen.de.
- Department for General-, Visceral-, Pediatric- and Transplant Surgery, University Clinic RWTH Aachen, Aachen, Germany. ddetemple@ukaachen.de.
- PRACTIS Clinician Scientist Program, Dean's Office for Academic Career Development, Hannover Medical School, Hannover, Germany. ddetemple@ukaachen.de.
- Department for General-, Visceral- and Transplant Surgery, Hannover Medical School, Hannover, Germany.
- Clinic for General- and Visceral Surgery, Agaplesion Ev. Klinikum Schaumburg, Obernkirchen, Germany.
- Clinic for General-, Visceral- and Oncological Surgery, Klinikum Bremen Mitte, Bremen, Germany.
Förderung
- Universitätsklinikum RWTH Aachen
Zitieren
APA
DeTemple, D. E., Störzer, S., Arbabzadah, S., Riddermann, A., Gronau, F., Timrott, K., Bektas, H., & Kleine, M. (2026). The role of large language models in the writing of surgical reviews: fact or fantasy?. Langenbeck's Archives of Surgery, 411(1). https://doi.org/10.1007/s00423-026-04095-2
Vancouver
DeTemple DE, Störzer S, Arbabzadah S, Riddermann A, Gronau F, Timrott K, et al. The role of large language models in the writing of surgical reviews: fact or fantasy?. Langenbeck's Archives of Surgery. 2026;411(1). doi:10.1007/s00423-026-04095-2
BibTeX
@article{DeTemple2026,
author = {DeTemple, Daphne E and Störzer, Simon and Arbabzadah, Sahar and Riddermann, Anna and Gronau, Felix and Timrott, Kai and Bektas, Hüseyin and Kleine, Moritz},
title = {The role of large language models in the writing of surgical reviews: fact or fantasy?},
journal = {Langenbeck's Archives of Surgery},
year = {2026},
volume = {411},
number = {1},
doi = {10.1007/s00423-026-04095-2}
}
EUVIMED