Asymptotic Confidence Interval, Sample Size Formulas and Comparison Test for the Agreement Intra‐Class Correlation Coefficient in Inter‐Rater Reliability Studies
Statistics in Medicine · 30.11.2024 · 43 (27) · 5060-5076
DOI: 10.1002/sim.10217 ↗Kurz gefasst
Vergleichsstudie in der Zeitschrift Statistics in Medicine, online erschienen am 16.09.2024. Verfasst von Abderrahmane Bourredjem und 2 weiteren Autor:innen. Beteiligte Einrichtung: Inserm CIC1432, Clinical Epidemiology Unit, Dijon, France. Hauptthemen laut MEDLINE: Computer Simulation, Observer Variation. Die Arbeit wurde bisher 7-mal zitiert (Crossref) und nennt 38 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0). Gefördert von: EIPHI Graduate School.
Abstract (Originalsprache)
The agreement intra‐class correlation coefficient (ICCa) is a suitable statistical index for inter‐rater reliability studies. With balanced Gaussian data, we prove the explicit form of ICCa asymptotic normality (ASN), valid both with analysis of variance (ANOVA), maximum likelihood (ML), or restricted ML (REML) estimates. An asymptotic confidence interval is then derived and its performances are examined by simulation compared to the most commonly used methods, under small, moderate and large sample size designs. Then, we deduce sample size calculation formulas, for the number of subjects and observers needed, to achieve a desired confidence interval width or an acceptable ICCa value test power and give concrete examples of their use. Finally, we propose a likelihood ratio test (LRT) to compare two ICCa's from two distinct subpopulations of patients (or raters) and study by simulation its first order risk and power properties. These methods are illustrated using data from two inter‐rater reliability studies, one in physiotherapy with 42 patients and 10 raters and the second in neonatology with 80 subjects and 14 raters. In conclusion, we made recommendations to employ the proposed confidence interval for medium to large samples combined with the quantification of the minimal required sample size at the planning step, or the posterior‐power at the analysis step, using simple dedicated formulas. Furthermore, with sufficient sizes, the proposed LRT seems suitable to compare inter‐rater reliability between two patient subpopulations. Used wisely, this proposed methods toolbox can remedy common current issues in inter‐rater reliability studies.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Inserm CIC1432, Clinical Epidemiology Unit, Dijon, France.
- Centre d'investigation Clinique, Module Epidémiologie Clinique/Essais Cliniques, Dijon-Bourgogne University Hospital, Dijon, France.
- Institut de Mathématiques de Bourgogne, UMR 5584, CNRS, Université de Bourgogne, Dijon, France.
- LAMOPS, ENSSEA, Tipasa, Algeria.
- Internal Medicine and Systemic Diseases Unit, University Hospital Dijon-Burgundy, Dijon, France.
ORCID-Profile: Abderrahmane Bourredjem
Förderung
- EIPHI Graduate School
Zitieren
APA
Bourredjem, A., Cardot, H., & Devilliers, H. (2024). Asymptotic Confidence Interval, Sample Size Formulas and Comparison Test for the Agreement Intra‐Class Correlation Coefficient in Inter‐Rater Reliability Studies. Statistics in Medicine, 43(27), 5060-5076. https://doi.org/10.1002/sim.10217
Vancouver
Bourredjem A, Cardot H, Devilliers H. Asymptotic Confidence Interval, Sample Size Formulas and Comparison Test for the Agreement Intra‐Class Correlation Coefficient in Inter‐Rater Reliability Studies. Statistics in Medicine. 2024;43(27):5060-5076. doi:10.1002/sim.10217
BibTeX
@article{Bourredjem2024,
author = {Bourredjem, Abderrahmane and Cardot, Hervé and Devilliers, Hervé},
title = {Asymptotic Confidence Interval, Sample Size Formulas and Comparison Test for the Agreement Intra‐Class Correlation Coefficient in Inter‐Rater Reliability Studies},
journal = {Statistics in Medicine},
year = {2024},
volume = {43},
number = {27},
pages = {5060--5076},
doi = {10.1002/sim.10217}
}
EUVIMED