EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

An empirical appraisal of eLife’s assessment vocabulary

Tom E Hardwicke iD, Sarah R Schiavone iD, Beth Clarke iD, Simine Vazire iD

PLOS Biology · 22.08.2024 · 22 (8) · e3002645

DOI: 10.1371/journal.pbio.3002645 ↗

Kurz gefasst

Fachartikel in der Zeitschrift PLOS Biology, online erschienen am 22.08.2024. Verfasst von Tom E Hardwicke und 3 weiteren Autor:innen. Beteiligte Einrichtung: Melbourne School of Psychological Sciences, University of Melbourne, Melbourne, Australia. Die Arbeit wurde bisher 3-mal zitiert (Crossref) und nennt 19 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Melbourne School of Psychological Sciences.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Research articles published by the journal eLife are accompanied by short evaluation statements that use phrases from a prescribed vocabulary to evaluate research on 2 dimensions: importance and strength of support. Intuitively, the prescribed phrases appear to be highly synonymous (e.g., important/valuable, compelling/convincing) and the vocabulary’s ordinal structure may not be obvious to readers. We conducted an online repeated-measures experiment to gauge whether the phrases were interpreted as intended. We also tested an alternative vocabulary with (in our view) a less ambiguous structure. A total of 301 participants with a doctoral or graduate degree used a 0% to 100% scale to rate the importance and strength of support of hypothetical studies described using phrases from both vocabularies. For the eLife vocabulary, most participants’ implied ranking did not match the intended ranking on both the importance ( n = 59, 20% matched, 95% confidence interval [15% to 24%]) and strength of support dimensions ( n = 45, 15% matched [11% to 20%]). By contrast, for the alternative vocabulary, most participants’ implied ranking did match the intended ranking on both the importance ( n = 188, 62% matched [57% to 68%]) and strength of support dimensions ( n = 201, 67% matched [62% to 72%]). eLife’s vocabulary tended to produce less consistent between-person interpretations, though the alternative vocabulary still elicited some overlapping interpretations away from the middle of the scale. We speculate that explicit presentation of a vocabulary’s intended ordinal structure could improve interpretation. Overall, these findings suggest that more structured and less ambiguous language can improve communication of research evaluations.

Autor:innen und Institutionen

  • Melbourne School of Psychological Sciences, University of Melbourne, Melbourne, Australia.

ORCID-Profile: Tom E Hardwicke, Sarah R Schiavone, Beth Clarke, Simine Vazire

Förderung

  • Melbourne School of Psychological Sciences

Zitieren

APA

Hardwicke, T. E., Schiavone, S. R., Clarke, B., & Vazire, S. (2024). An empirical appraisal of eLife’s assessment vocabulary. PLOS Biology, 22(8), e3002645. https://doi.org/10.1371/journal.pbio.3002645

Vancouver

Hardwicke TE, Schiavone SR, Clarke B, Vazire S. An empirical appraisal of eLife’s assessment vocabulary. PLOS Biology. 2024;22(8):e3002645. doi:10.1371/journal.pbio.3002645

BibTeX
@article{Hardwicke2024,
  author = {Hardwicke, Tom E and Schiavone, Sarah R and Clarke, Beth and Vazire, Simine},
  title = {An empirical appraisal of eLife’s assessment vocabulary},
  journal = {PLOS Biology},
  year = {2024},
  volume = {22},
  number = {8},
  pages = {e3002645},
  doi = {10.1371/journal.pbio.3002645}
}
Kostenlos starten →