Robust Permutation Test of Intraclass Correlation Coefficient for Assessing Agreement
Cancers · 21.08.2025 · 17 (16) · 2713
DOI: 10.3390/cancers17162713 ↗Kurz gefasst
Fachartikel in der Zeitschrift Cancers, online erschienen am 21.08.2025. Verfasst von Mengyu Fang und 2 weiteren Autor:innen. Beteiligte Einrichtung: Department of Biostatistics and Bioinformatics, Roswell Park Comprehensive Cancer Center, Buffalo, NY 14263, USA. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 24 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: NCI NIH HHS, Roswell Park Cancer Institute and National Cancer Institute (NCI), NCI NRG Oncology Statistical and Data Management Center.
Abstract (Originalsprache)
Background
Inter-rater reliability is critical in oncology to ensure consistent and reliable measurements across raters and methods, such as when evaluating biomarker levels in different laboratories or comparing tumor size assessments by radiation oncologists during therapy planning. This consistency is essential for informed decision-making in both clinical and research contexts, and the intraclass correlation coefficient (ICC) is a widely recommended statistic for assessing agreement. This work focuses on hypothesis testing of the ICC(2,1) with two raters.
Methods
We evaluated the performance of a naive permutation test for testing the hypothesis H0:ICC=0 and found that it fails to reliably control the type I error rate. To address this, we developed a robust permutation test based on a studentized statistic, which we prove to be asymptotically valid even when paired variables are uncorrelated but dependent.
Results
Simulation studies demonstrate that the proposed test consistently maintains type I error control, even with small sample sizes, outperforming the naive approach across various data-generating scenarios.
Conclusions
The proposed studentized permutation test for ICC(2,1) offers a statistically valid and robust method for assessing inter-rater reliability and demonstrates practical utility when applied to two real-world oncology datasets.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Biostatistics and Bioinformatics, Roswell Park Comprehensive Cancer Center, Buffalo, NY 14263, USA.
ORCID-Profile: Alan David Hutson
Förderung
- NCI NIH HHS
- Roswell Park Cancer Institute and National Cancer Institute (NCI)
- NCI NRG Oncology Statistical and Data Management Center
- NCI IOTN Moonshot
- NCI ARTNet Moonshot
- NCI CAP-IT
Zitieren
APA
Fang, M., Hutson, A. D., & Yu, H. (2025). Robust Permutation Test of Intraclass Correlation Coefficient for Assessing Agreement. Cancers, 17(16), 2713. https://doi.org/10.3390/cancers17162713
Vancouver
Fang M, Hutson AD, Yu H. Robust Permutation Test of Intraclass Correlation Coefficient for Assessing Agreement. Cancers. 2025;17(16):2713. doi:10.3390/cancers17162713
BibTeX
@article{Fang2025,
author = {Fang, Mengyu and Hutson, Alan David and Yu, Han},
title = {Robust Permutation Test of Intraclass Correlation Coefficient for Assessing Agreement},
journal = {Cancers},
year = {2025},
volume = {17},
number = {16},
pages = {2713},
doi = {10.3390/cancers17162713}
}
EUVIMED