EUVIMEDEUVIMED entdecken
BeobachtungsstudieOpen Access · CC BY 4.0

Cite-seeing and reviewing: A study on citation bias in peer review

Ivan Stelmakh iD, Charvi Rastogi iD, Ryan Liu iD, Shuchi Chawla, Federico Echenique, Nihar B Shah

PLOS ONE · 2023

DOI: 10.1371/journal.pone.0283980 ↗

Kurz gefasst

Beobachtungsstudie in der Zeitschrift PLOS ONE, 2023. Verfasst von Ivan Stelmakh und 5 weiteren Autor:innen. Beteiligte Einrichtung: New Economic School, Moscow, Russia. Hauptthemen laut MEDLINE: Peer Review, Research Personnel. Die Arbeit wurde bisher 19-mal zitiert (Crossref) und nennt 32 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Science Foundation, J.P. Morgan AI research fellowship.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Citations play an important role in researchers’ careers as a key factor in evaluation of scientific impact. Many anecdotes advice authors to exploit this fact and cite prospective reviewers to try obtaining a more positive evaluation for their submission. In this work, we investigate if such a citation bias actually exists: Does the citation of a reviewer’s own work in a submission cause them to be positively biased towards the submission? In conjunction with the review process of two flagship conferences in machine learning and algorithmic economics, we execute an observational study to test for citation bias in peer review. In our analysis, we carefully account for various confounding factors such as paper quality and reviewer expertise, and apply different modeling techniques to alleviate concerns regarding the model mismatch. Overall, our analysis involves 1,314 papers and 1,717 reviewers and detects citation bias in both venues we consider. In terms of the effect size, by citing a reviewer’s work, a submission has a non-trivial chance of getting a higher score from the reviewer: an expected increase in the score is approximately 0.23 on a 5-point Likert item. For reference, a one-point increase of a score by a single reviewer improves the position of a submission by 11% on average.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Peer ReviewResearch PersonnelHumansProspective StudiesBiasMachine LearningPeer Review, Research

Autor:innen und Institutionen

  • New Economic School, Moscow, Russia.
  • Yakov & Partners, Moscow, Russia.
  • School of Computer Science, Carnegie Mellon University, Pittsburgh, Pennsylvania, United States of America.
  • Department of Computer Science, University of Texas at Austin, Austin, Texas, United States of America.
  • Division of the Humanities and Social Sciences, California Institute of Technology, Pasadena, California, United States of America.

ORCID-Profile: Ivan Stelmakh, Charvi Rastogi, Ryan Liu

Förderung

  • National Science Foundation
  • J.P. Morgan AI research fellowship

Zitieren

APA

Stelmakh, I., Rastogi, C., Liu, R., Chawla, S., Echenique, F., & Shah, N. B. (2023). Cite-seeing and reviewing: A study on citation bias in peer review. PLOS ONE. https://doi.org/10.1371/journal.pone.0283980

Vancouver

Stelmakh I, Rastogi C, Liu R, Chawla S, Echenique F, Shah NB. Cite-seeing and reviewing: A study on citation bias in peer review. PLOS ONE. 2023. doi:10.1371/journal.pone.0283980

BibTeX
@article{Stelmakh2023,
  author = {Stelmakh, Ivan and Rastogi, Charvi and Liu, Ryan and Chawla, Shuchi and Echenique, Federico and Shah, Nihar B},
  title = {Cite-seeing and reviewing: A study on citation bias in peer review},
  journal = {PLOS ONE},
  year = {2023},
  doi = {10.1371/journal.pone.0283980}
}
Kostenlos starten →