EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Comparison of DeepSeek AI and Human Reviewers in Structured Evaluation of Neurosurgical Manuscripts: A Pilot Study

Kishore Balasubramanian iD, Daniel O'Connor, Beste Gülsuna, Visish M Srinivasan, Daniel A Donoho, Nicholas Sader, Christopher S Graffeo iD

Journal of Neurological Surgery Reports · 01.10.2026 · 87 (04) · e192-e201

DOI: 10.1055/a-2964-9336 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of Neurological Surgery Reports, erschienen am 01.10.2026. Verfasst von Kishore Balasubramanian und 6 weiteren Autor:innen. Beteiligte Einrichtung: Department of NeurosurgeryThe University of Oklahoma College of MedicineOklahoma CityOklahomaUnited States. Die Arbeit nennt 24 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Objectives

Increasing manuscript volume has strained peer review, prompting interest in artificial intelligence (AI) as a supportive tool. We evaluated DeepSeek R1 as a structured peer reviewer of neurosurgical case-based manuscripts and compared its scoring patterns with those of an experienced editor and student reviewers.

Design

Exploratory, cross-sectional comparative study using a convenience sample of consecutively submitted manuscripts.

Setting

Single skull-base neurosurgical journal during a 6-month submission window (January–June 2025).

Participants

Three reviewer groups independently assessed 11 de-identified, unpublished neurosurgical manuscripts: first-year medical students (n = 154; each randomly assigned one manuscript after a course in biostatistics, epidemiology, and critical appraisal), the journal’s Editor-in-Chief (Experienced Reviewer), and DeepSeek R1. Main Outcome Measures Each manuscript was rated on a 3-point Likert scale (1 = low, 2 = intermediate, 3 = high) across five domains: Study Objective, Novelty, Study Design, Thoroughness, and Clarity.

Results

Significant between-group differences were found in four of five domains: Study Objective (H = 8.31, p = 0.02), Novelty (H = 9.74, p = 0.008), Thoroughness (H = 10.49, p = 0.005), and Clarity (H = 8.13, p = 0.02); Study Design did not differ (H = 1.45, p = 0.483). DeepSeek scored significantly higher than the Experienced Reviewer for Study Objective (p = 0.015) and higher than both human groups for Novelty.

Conclusions

DeepSeek showed consistency in structured domains but systematically inflated subjective ratings, particularly Novelty. A human-in-the-loop model integrating AI efficiency with expert judgment may optimize peer review.

Schlagwörter

Stichwörter der Autor:innen

DeepSeekartificial intelligencelarge language modelneurosurgerypeer review

Autor:innen und Institutionen

  • Department of NeurosurgeryThe University of Oklahoma College of MedicineOklahoma CityOklahomaUnited States.
  • Department of NeurosurgeryUniversity of Pennsylvania Perelman School of MedicinePhiladelphiaPennsylvaniaUnited States.
  • Division of NeurosurgeryChildren's National Medical CenterWashingtonDistrict of ColumbiaUnited States.
  • Department of NeurosurgeryUniversity of Oklahoma Health Sciences CenterOklahoma CityOklahomaUnited States.

ORCID-Profile: Kishore Balasubramanian, Christopher S Graffeo

Zitieren

APA

Balasubramanian, K., O'Connor, D., Gülsuna, B., Srinivasan, V. M., Donoho, D. A., Sader, N., & Graffeo, C. S. (2026). Comparison of DeepSeek AI and Human Reviewers in Structured Evaluation of Neurosurgical Manuscripts: A Pilot Study. Journal of Neurological Surgery Reports, 87(04), e192-e201. https://doi.org/10.1055/a-2964-9336

Vancouver

Balasubramanian K, O'Connor D, Gülsuna B, Srinivasan VM, Donoho DA, Sader N, et al. Comparison of DeepSeek AI and Human Reviewers in Structured Evaluation of Neurosurgical Manuscripts: A Pilot Study. Journal of Neurological Surgery Reports. 2026;87(04):e192-e201. doi:10.1055/a-2964-9336

BibTeX
@article{Balasubramanian2026,
  author = {Balasubramanian, Kishore and O'Connor, Daniel and Gülsuna, Beste and Srinivasan, Visish M and Donoho, Daniel A and Sader, Nicholas and Graffeo, Christopher S},
  title = {Comparison of DeepSeek AI and Human Reviewers in Structured Evaluation of Neurosurgical Manuscripts: A Pilot Study},
  journal = {Journal of Neurological Surgery Reports},
  year = {2026},
  volume = {87},
  number = {04},
  pages = {e192--e201},
  doi = {10.1055/a-2964-9336}
}
Kostenlos starten →