EUVIMEDDescoperă EUVIMED
ÜbersichtsarbeitOpen access · CC BY 4.0

Confidence Intervals and Sample Size for the ICC in Two‐Way ANOVA Models

Dipro Mondal iD, Math J J M Candel iD, Alberto Cassese iD, Sophie Vanbelle iD

Statistics in Medicine · 2025-05 · 44 (10-12)

DOI: 10.1002/sim.70106 ↗

At a glance

Übersichtsarbeit in Statistics in Medicine, published online on 2025-05-22. Written by Dipro Mondal and 3 co-authors. Institution: Department of Methodology and Statistics, Care and Public Health Research Institute (CAPHRI), Maastricht University, Maastricht, the Netherlands. Major MEDLINE topics: Models, Statistical. Cited 9 times so far (Crossref); 46 references. The full text is freely available (open access, CC BY 4.0). Funded by: Nederlandse Organisatie voor Wetenschappelijk Onderzoek.

RezumatSubjectsDate bibliograficeCiteRelated studies

Abstract (original language)

The reliability of measurement instruments is vital in fields like medicine and psychology, where these tools are often used for diagnostic purposes. In reliability studies where participants are assessed by the same set of raters, the data can be modeled using a two‐way ANOVA, with the intraclass correlation coefficient (ICC) as a key metric. This paper focuses on the ICC for agreement, which is crucial when the measurement values themselves, rather than just their rank ordering, are of interest. However, selecting appropriate confidence interval methods and determining adequate sample sizes for the ICC for agreement remains challenging. This work advances the understanding of confidence interval methods for the ICC for agreement, provides practical tools, and offers recommendations for selecting confidence interval methods and sample size procedures for planning reliability studies. In particular, we provide a comprehensive review and simulation‐based comparison of six classes of confidence interval methods for the ICC for agreement identified in the literature. Our analysis includes a method based on the F‐distribution, previously omitted, which demonstrates the best statistical properties in some cases. Then, in conjunction with the best‐performing methods, we evaluate three sample size determination procedures based on the expected width of confidence intervals that we identified in the literature. To address the lack of accessible tools, we further developed an interactive R Shiny app, freely available to researchers, to compute confidence intervals and sample sizes. The utility of these methods is illustrated by a study on fetal heart rates.

Subjects

MeSH terms (MEDLINE)

Models, StatisticalSample SizeHumansConfidence IntervalsReproducibility of ResultsAnalysis of VarianceComputer SimulationData Interpretation, Statistical

Author keywords

interrater reliabilityintrarater reliabilitymeasurement errorsobserver variationreproducibility of results

Authors and institutions

  • Department of Methodology and Statistics, Care and Public Health Research Institute (CAPHRI), Maastricht University, Maastricht, the Netherlands.
  • Department of Statistics, Computer Science, Applications "Giuseppe Parenti" The University of Florence, Firenze, Italy.

ORCID profiles: Dipro Mondal, Math J J M Candel, Alberto Cassese, Sophie Vanbelle

Funding

  • Nederlandse Organisatie voor Wetenschappelijk Onderzoek

Cite

APA

Mondal, D., Candel, M. J. J. M., Cassese, A., & Vanbelle, S. (2025). Confidence Intervals and Sample Size for the ICC in Two‐Way ANOVA Models. Statistics in Medicine, 44(10-12). https://doi.org/10.1002/sim.70106

Vancouver

Mondal D, Candel MJJM, Cassese A, Vanbelle S. Confidence Intervals and Sample Size for the ICC in Two‐Way ANOVA Models. Statistics in Medicine. 2025;44(10-12). doi:10.1002/sim.70106

BibTeX
@article{Mondal2025,
  author = {Mondal, Dipro and Candel, Math J J M and Cassese, Alberto and Vanbelle, Sophie},
  title = {Confidence Intervals and Sample Size for the ICC in Two‐Way ANOVA Models},
  journal = {Statistics in Medicine},
  year = {2025},
  volume = {44},
  number = {10-12},
  doi = {10.1002/sim.70106}
}
Start for free →