Comparison of DeepSeek AI and Human Reviewers in Structured Evaluation of Neurosurgical Manuscripts: A Pilot Study
Journal of Neurological Surgery Reports · 01.10.2026 · 87 (04) · e192-e201
DOI: 10.1055/a-2964-9336 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Neurological Surgery Reports, erschienen am 01.10.2026. Verfasst von Kishore Balasubramanian und 6 weiteren Autor:innen. Beteiligte Einrichtung: Department of NeurosurgeryThe University of Oklahoma College of MedicineOklahoma CityOklahomaUnited States. Die Arbeit nennt 24 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Objectives
Increasing manuscript volume has strained peer review, prompting interest in artificial intelligence (AI) as a supportive tool. We evaluated DeepSeek R1 as a structured peer reviewer of neurosurgical case-based manuscripts and compared its scoring patterns with those of an experienced editor and student reviewers.
Design
Exploratory, cross-sectional comparative study using a convenience sample of consecutively submitted manuscripts.
Setting
Single skull-base neurosurgical journal during a 6-month submission window (January–June 2025).
Participants
Three reviewer groups independently assessed 11 de-identified, unpublished neurosurgical manuscripts: first-year medical students (n = 154; each randomly assigned one manuscript after a course in biostatistics, epidemiology, and critical appraisal), the journal’s Editor-in-Chief (Experienced Reviewer), and DeepSeek R1. Main Outcome Measures Each manuscript was rated on a 3-point Likert scale (1 = low, 2 = intermediate, 3 = high) across five domains: Study Objective, Novelty, Study Design, Thoroughness, and Clarity.
Results
Significant between-group differences were found in four of five domains: Study Objective (H = 8.31, p = 0.02), Novelty (H = 9.74, p = 0.008), Thoroughness (H = 10.49, p = 0.005), and Clarity (H = 8.13, p = 0.02); Study Design did not differ (H = 1.45, p = 0.483). DeepSeek scored significantly higher than the Experienced Reviewer for Study Objective (p = 0.015) and higher than both human groups for Novelty.
Conclusions
DeepSeek showed consistency in structured domains but systematically inflated subjective ratings, particularly Novelty. A human-in-the-loop model integrating AI efficiency with expert judgment may optimize peer review.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of NeurosurgeryThe University of Oklahoma College of MedicineOklahoma CityOklahomaUnited States.
- Department of NeurosurgeryUniversity of Pennsylvania Perelman School of MedicinePhiladelphiaPennsylvaniaUnited States.
- Division of NeurosurgeryChildren's National Medical CenterWashingtonDistrict of ColumbiaUnited States.
- Department of NeurosurgeryUniversity of Oklahoma Health Sciences CenterOklahoma CityOklahomaUnited States.
ORCID-Profile: Kishore Balasubramanian, Christopher S Graffeo
Zitieren
APA
Balasubramanian, K., O'Connor, D., Gülsuna, B., Srinivasan, V. M., Donoho, D. A., Sader, N., & Graffeo, C. S. (2026). Comparison of DeepSeek AI and Human Reviewers in Structured Evaluation of Neurosurgical Manuscripts: A Pilot Study. Journal of Neurological Surgery Reports, 87(04), e192-e201. https://doi.org/10.1055/a-2964-9336
Vancouver
Balasubramanian K, O'Connor D, Gülsuna B, Srinivasan VM, Donoho DA, Sader N, et al. Comparison of DeepSeek AI and Human Reviewers in Structured Evaluation of Neurosurgical Manuscripts: A Pilot Study. Journal of Neurological Surgery Reports. 2026;87(04):e192-e201. doi:10.1055/a-2964-9336
BibTeX
@article{Balasubramanian2026,
author = {Balasubramanian, Kishore and O'Connor, Daniel and Gülsuna, Beste and Srinivasan, Visish M and Donoho, Daniel A and Sader, Nicholas and Graffeo, Christopher S},
title = {Comparison of DeepSeek AI and Human Reviewers in Structured Evaluation of Neurosurgical Manuscripts: A Pilot Study},
journal = {Journal of Neurological Surgery Reports},
year = {2026},
volume = {87},
number = {04},
pages = {e192--e201},
doi = {10.1055/a-2964-9336}
}
EUVIMED