An empirical appraisal of eLife’s assessment vocabulary
PLOS Biology · 22.08.2024 · 22 (8) · e3002645
DOI: 10.1371/journal.pbio.3002645 ↗Kurz gefasst
Fachartikel in der Zeitschrift PLOS Biology, online erschienen am 22.08.2024. Verfasst von Tom E Hardwicke und 3 weiteren Autor:innen. Beteiligte Einrichtung: Melbourne School of Psychological Sciences, University of Melbourne, Melbourne, Australia. Die Arbeit wurde bisher 3-mal zitiert (Crossref) und nennt 19 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Melbourne School of Psychological Sciences.
Abstract (Originalsprache)
Research articles published by the journal eLife are accompanied by short evaluation statements that use phrases from a prescribed vocabulary to evaluate research on 2 dimensions: importance and strength of support. Intuitively, the prescribed phrases appear to be highly synonymous (e.g., important/valuable, compelling/convincing) and the vocabulary’s ordinal structure may not be obvious to readers. We conducted an online repeated-measures experiment to gauge whether the phrases were interpreted as intended. We also tested an alternative vocabulary with (in our view) a less ambiguous structure. A total of 301 participants with a doctoral or graduate degree used a 0% to 100% scale to rate the importance and strength of support of hypothetical studies described using phrases from both vocabularies. For the eLife vocabulary, most participants’ implied ranking did not match the intended ranking on both the importance ( n = 59, 20% matched, 95% confidence interval [15% to 24%]) and strength of support dimensions ( n = 45, 15% matched [11% to 20%]). By contrast, for the alternative vocabulary, most participants’ implied ranking did match the intended ranking on both the importance ( n = 188, 62% matched [57% to 68%]) and strength of support dimensions ( n = 201, 67% matched [62% to 72%]). eLife’s vocabulary tended to produce less consistent between-person interpretations, though the alternative vocabulary still elicited some overlapping interpretations away from the middle of the scale. We speculate that explicit presentation of a vocabulary’s intended ordinal structure could improve interpretation. Overall, these findings suggest that more structured and less ambiguous language can improve communication of research evaluations.
Autor:innen und Institutionen
- Melbourne School of Psychological Sciences, University of Melbourne, Melbourne, Australia.
ORCID-Profile: Tom E Hardwicke, Sarah R Schiavone, Beth Clarke, Simine Vazire
Förderung
- Melbourne School of Psychological Sciences
Zitieren
APA
Hardwicke, T. E., Schiavone, S. R., Clarke, B., & Vazire, S. (2024). An empirical appraisal of eLife’s assessment vocabulary. PLOS Biology, 22(8), e3002645. https://doi.org/10.1371/journal.pbio.3002645
Vancouver
Hardwicke TE, Schiavone SR, Clarke B, Vazire S. An empirical appraisal of eLife’s assessment vocabulary. PLOS Biology. 2024;22(8):e3002645. doi:10.1371/journal.pbio.3002645
BibTeX
@article{Hardwicke2024,
author = {Hardwicke, Tom E and Schiavone, Sarah R and Clarke, Beth and Vazire, Simine},
title = {An empirical appraisal of eLife’s assessment vocabulary},
journal = {PLOS Biology},
year = {2024},
volume = {22},
number = {8},
pages = {e3002645},
doi = {10.1371/journal.pbio.3002645}
}
EUVIMED