ChatGPT identifies gender disparities in scientific peer review
eLife · 2023
DOI: 10.7554/elife.90230 ↗Kurz gefasst
Fachartikel in der Zeitschrift eLife, 2023. Verfasst von Jeroen P H Verharen. Beteiligte Einrichtung: Department of Molecular and Cell Biology and Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, United States. Hauptthemen laut MEDLINE: Publishing, Artificial Intelligence. Die Arbeit wurde bisher 15-mal zitiert (Crossref) und nennt 30 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
The peer review process is a critical step in ensuring the quality of scientific research. However, its subjectivity has raised concerns. To investigate this issue, I examined over 500 publicly available peer review reports from 200 published neuroscience papers in 2022–2023. OpenAI’s generative artificial intelligence ChatGPT was used to analyze language use in these reports, which demonstrated superior performance compared to traditional lexicon- and rule-based language models. As expected, most reviews for these published papers were seen as favorable by ChatGPT (89.8% of reviews), and language use was mostly polite (99.8% of reviews). However, this analysis also demonstrated high levels of variability in how each reviewer scored the same paper, indicating the presence of subjectivity in the peer review process. The results further revealed that female first authors received less polite reviews than their male peers, indicating a gender bias in reviewing. In addition, published papers with a female senior author received more favorable reviews than papers with a male senior author, for which I discuss potential causes. Together, this study highlights the potential of generative artificial intelligence in performing natural language processing of specialized scientific texts. As a proof of concept, I show that ChatGPT can identify areas of concern in scientific peer review, underscoring the importance of transparent peer review in studying equitability in scientific publishing.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Molecular and Cell Biology and Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, United States.
ORCID-Profile: Jeroen P H Verharen
Zitieren
APA
Verharen, J. P. H. (2023). ChatGPT identifies gender disparities in scientific peer review. eLife. https://doi.org/10.7554/elife.90230
Vancouver
Verharen JPH. ChatGPT identifies gender disparities in scientific peer review. eLife. 2023. doi:10.7554/elife.90230
BibTeX
@article{Verharen2023,
author = {Verharen, Jeroen P H},
title = {ChatGPT identifies gender disparities in scientific peer review},
journal = {eLife},
year = {2023},
doi = {10.7554/elife.90230}
}
EUVIMED