EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Robust Permutation Test of Intraclass Correlation Coefficient for Assessing Agreement

Mengyu Fang, Alan David Hutson iD, Han Yu

Cancers · 21.08.2025 · 17 (16) · 2713

DOI: 10.3390/cancers17162713 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Cancers, online erschienen am 21.08.2025. Verfasst von Mengyu Fang und 2 weiteren Autor:innen. Beteiligte Einrichtung: Department of Biostatistics and Bioinformatics, Roswell Park Comprehensive Cancer Center, Buffalo, NY 14263, USA. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 24 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: NCI NIH HHS, Roswell Park Cancer Institute and National Cancer Institute (NCI), NCI NRG Oncology Statistical and Data Management Center.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Inter-rater reliability is critical in oncology to ensure consistent and reliable measurements across raters and methods, such as when evaluating biomarker levels in different laboratories or comparing tumor size assessments by radiation oncologists during therapy planning. This consistency is essential for informed decision-making in both clinical and research contexts, and the intraclass correlation coefficient (ICC) is a widely recommended statistic for assessing agreement. This work focuses on hypothesis testing of the ICC(2,1) with two raters.

Methods

We evaluated the performance of a naive permutation test for testing the hypothesis H0:ICC=0 and found that it fails to reliably control the type I error rate. To address this, we developed a robust permutation test based on a studentized statistic, which we prove to be asymptotically valid even when paired variables are uncorrelated but dependent.

Results

Simulation studies demonstrate that the proposed test consistently maintains type I error control, even with small sample sizes, outperforming the naive approach across various data-generating scenarios.

Conclusions

The proposed studentized permutation test for ICC(2,1) offers a statistically valid and robust method for assessing inter-rater reliability and demonstrates practical utility when applied to two real-world oncology datasets.

Schlagwörter

Stichwörter der Autor:innen

ICCagreementpermutation test

Autor:innen und Institutionen

  • Department of Biostatistics and Bioinformatics, Roswell Park Comprehensive Cancer Center, Buffalo, NY 14263, USA.

ORCID-Profile: Alan David Hutson

Förderung

  • NCI NIH HHS
  • Roswell Park Cancer Institute and National Cancer Institute (NCI)
  • NCI NRG Oncology Statistical and Data Management Center
  • NCI IOTN Moonshot
  • NCI ARTNet Moonshot
  • NCI CAP-IT

Zitieren

APA

Fang, M., Hutson, A. D., & Yu, H. (2025). Robust Permutation Test of Intraclass Correlation Coefficient for Assessing Agreement. Cancers, 17(16), 2713. https://doi.org/10.3390/cancers17162713

Vancouver

Fang M, Hutson AD, Yu H. Robust Permutation Test of Intraclass Correlation Coefficient for Assessing Agreement. Cancers. 2025;17(16):2713. doi:10.3390/cancers17162713

BibTeX
@article{Fang2025,
  author = {Fang, Mengyu and Hutson, Alan David and Yu, Han},
  title = {Robust Permutation Test of Intraclass Correlation Coefficient for Assessing Agreement},
  journal = {Cancers},
  year = {2025},
  volume = {17},
  number = {16},
  pages = {2713},
  doi = {10.3390/cancers17162713}
}
Kostenlos starten →