EUVIMEDEUVIMED entdecken
Crossref

Variation and inter‐rater reliability of abstract reviewer scores for clinical vignettes submitted to a national hospitalist meeting

John R Stephens, Andrew Morgan, Nelly Bellamy, Evan J Raff, Leonard Feldman iD

Journal of Hospital Medicine · 2026-05 · 21 (5) · 520-524

DOI: 10.1002/jhm.70261 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of Hospital Medicine, online erschienen am 22.01.2026. Verfasst von John R Stephens und 4 weiteren Autor:innen. Beteiligte Einrichtung: Department of Medicine, Division of Hospital Medicine, University of North Carolina School of Medicine, Chapel Hill, USA. Hauptthemen laut MEDLINE: Hospitalists, Peer Review, Research, Congresses as Topic. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 17 Literaturangaben.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Peer review of research products suffers from poor inter‐rater reliability. Few studies examine whether this limitation generalizes to case reports. We conducted a cross‐sectional analysis of peer reviews of clinical vignette abstracts submitted to a national hospitalist meeting in 2024 and 2025. Three randomly assigned reviewers scored each vignette on a 1–10 scale. We analyzed variation in scores across abstracts and reviewers and estimated inter‐rater reliability via intraclass correlation coefficient (ICC). Two hundred twenty‐one reviewers evaluated 1630 abstracts in 2024–2025. Abstract scores varied substantially: 384/1630 (23.6%) abstracts had a difference of 4 or more points (>2 standard deviations) between highest and lowest reviewer scores. Scores varied by reviewer: 2024 reviewer‐level mean scores ranged 4.27–8.47 (standard deviation (SD): 0.70–2.80); 2025 scores ranged 4.06–8.59 (SD: 0.62‐2.69). Inter‐rater reliability was poor (ICC: 0.37). Adjusting final scores based on reviewer scoring tendencies changed the accept/reject category for 183 (11.2%) abstracts, suggesting opportunities for quality improvement.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

HospitalistsPeer Review, ResearchCongresses as TopicHumansReproducibility of ResultsCross-Sectional StudiesObserver Variation

Autor:innen und Institutionen

  • Department of Medicine, Division of Hospital Medicine, University of North Carolina School of Medicine, Chapel Hill, USA.
  • Departments of Medicine and Pediatrics, Division of Hospital Medicine, Johns Hopkins School of Medicine, Baltimore, Maryland, USA.

ORCID-Profile: Leonard Feldman

Zitieren

APA

Stephens, J. R., Morgan, A., Bellamy, N., Raff, E. J., & Feldman, L. (2026). Variation and inter‐rater reliability of abstract reviewer scores for clinical vignettes submitted to a national hospitalist meeting. Journal of Hospital Medicine, 21(5), 520-524. https://doi.org/10.1002/jhm.70261

Vancouver

Stephens JR, Morgan A, Bellamy N, Raff EJ, Feldman L. Variation and inter‐rater reliability of abstract reviewer scores for clinical vignettes submitted to a national hospitalist meeting. Journal of Hospital Medicine. 2026;21(5):520-524. doi:10.1002/jhm.70261

BibTeX
@article{Stephens2026,
  author = {Stephens, John R and Morgan, Andrew and Bellamy, Nelly and Raff, Evan J and Feldman, Leonard},
  title = {Variation and inter‐rater reliability of abstract reviewer scores for clinical vignettes submitted to a national hospitalist meeting},
  journal = {Journal of Hospital Medicine},
  year = {2026},
  volume = {21},
  number = {5},
  pages = {520--524},
  doi = {10.1002/jhm.70261}
}
Kostenlos starten →