EUVIMEDDécouvrir EUVIMED
VergleichsstudieOpen access · CC BY-NC-ND 4.0

A Neutral Comparison of Multivariate Hypothesis Tests for Parallel Group Designs in Rare Disease Settings

Martin Geroldinger iD, Martin Laimer, Verena Wally iD, Johann W Bauer iD, Georg Zimmermann iD

Statistics in Medicine · 2026-09 · 45 (20-22)

DOI: 10.1002/sim.70735 ↗

At a glance

Vergleichsstudie in Statistics in Medicine, published online on 2026-09-21. Written by Martin Geroldinger and 4 co-authors. Institution: Research Program Biomedical Data Science, Paracelsus Medical University, Salzburg, Austria. Major MEDLINE topics: Rare Diseases. Cited 1 times so far (Crossref); 24 references. The full text is freely available (open access, CC BY-NC-ND 4.0). Funded by: Salzburger Landesregierung (WISS 2025 project"servEB"), Innovative Health Initiative, HORIZON-HLTH-2023-DISEASE-07-01.

RésuméSubjectsRéférences bibliographiquesCiteRelated studies

Abstract (original language)

Multivariate endpoints are increasingly used in clinical studies on rare diseases, where sample sizes tend to be small and distributional assumptions are often violated. In such settings, commonly applied parametric multivariate methods, such as classical MANOVA approaches, may suffer from inflated type‐I error or reduced statistical power. Therefore, a neutral simulation‐based comparison of four methodologically different approaches for the analysis of multivariate outcomes in parallel group designs was conducted. The four approaches included the nonparametric R package npmv (using permutation and randomization resampling for small samples), a semiparametric MANOVA approach from the R package MANOVA.RM with both a Wald‐type statistics (WTS) and a modified ANOVA‐type statistics (MATS) using all available variants including also bootstrap approaches, a classical parametric MANOVA , and multiple univariate Mann–Whitney tests with Holm correction. The simulation framework was informed by empirical distributions derived from anonymized clinical data and clinical expert guidance. Two clinically realistic effect types were mainly evaluated: (i) percent‐responder effects for wound size reduction and (ii) additional location shifts in pain and pruritus visual analog scales (VAS) scores. In addition, scenarios are examined in which classical assumptions such as multivariate normality and homogeneity of covariance matrices are violated. The sample sizes considered ranged from 12 to 360 subjects. Both balanced and unbalanced designs were evaluated, with the objective of illustrating potential limitations of methods when applied to small samples. Outcomes were assessed in terms of type‐I error control and statistical power. Overall, npmv , the MATS using especially parametric bootstrap within the semiparametric MANOVA.RM package maintained reliable type‐I error control, whereas the semiparametric MANOVA.RM approach with the WTS using its asymptotic approximation showed very liberal type‐I error behavior to almost no type‐I error control in small sample scenarios, and especially in unbalanced settings. Bootstrap approaches especially of the WTS substantially improved type‐I error control compared with the asymptotic WTS. The nonparametric npmv approach achieved good power values, especially in small samples, as did the modified ATS approach using parametric bootstrap from the MANOVA.RM package, which achieved very similar results. The univariate Mann–Whitney U test with Holm correction tended to yield conservative results for small samples; however, despite the univariate nature of this approach, it achieved a high and comparable level of statistical power in many scenarios. Classical MANOVA also yielded good results in certain scenarios; however, it exhibited rather liberal type‐I error rates, primarily in heteroscedastic scenarios and unbalanced designs, which can be explained by a violation of the assumption of homoscedastic covariance matrices. These results highlight the strengths and limitations of a multi‐aspect analysis in parallel group designs especially for small sample settings, as common in rare diseases, and provide guidance for the analysis in rare disease research.

Subjects

MeSH terms (MEDLINE)

Rare DiseasesHumansMultivariate AnalysisComputer SimulationModels, StatisticalSample SizeResearch Design

Author keywords

MANOVA.RMmultivariate statisticsmulti‐aspect modelingneutral comparisonnpmvparallel‐group designrare diseases

Authors and institutions

  • Research Program Biomedical Data Science, Paracelsus Medical University, Salzburg, Austria.
  • Team Biostatistics and Big Medical Data, IDA Lab Salzburg, Paracelsus Medical University, Salzburg, Austria.
  • Department of Neurology, Christian Doppler Medical Centre, Full Member of European Reference Network on Rare and Complex Epilepsies EpiCARE, Paracelsus Medical University, Salzburg, Austria.
  • Department of Dermatology and Allergology, Paracelsus Medical University, Salzburg, Austria.
  • EB House Austria, Research Program for Molecular Therapy of Genodermatoses, Department of Dermatology and Allergology, University Hospital of the Paracelsus Medical University Salzburg, Salzburg, Austria.
  • EB Research Institute GmbH, Salzburg, Austria.

ORCID profiles: Martin Geroldinger, Verena Wally, Johann W Bauer, Georg Zimmermann

Funding

  • Salzburger Landesregierung (WISS 2025 project"servEB")
  • Innovative Health Initiative
  • HORIZON-HLTH-2023-DISEASE-07-01
  • IDA-Lab Salzburg
  • EXDIGIT

Cite

APA

Geroldinger, M., Laimer, M., Wally, V., Bauer, J. W., & Zimmermann, G. (2026). A Neutral Comparison of Multivariate Hypothesis Tests for Parallel Group Designs in Rare Disease Settings. Statistics in Medicine, 45(20-22). https://doi.org/10.1002/sim.70735

Vancouver

Geroldinger M, Laimer M, Wally V, Bauer JW, Zimmermann G. A Neutral Comparison of Multivariate Hypothesis Tests for Parallel Group Designs in Rare Disease Settings. Statistics in Medicine. 2026;45(20-22). doi:10.1002/sim.70735

BibTeX
@article{Geroldinger2026,
  author = {Geroldinger, Martin and Laimer, Martin and Wally, Verena and Bauer, Johann W and Zimmermann, Georg},
  title = {A Neutral Comparison of Multivariate Hypothesis Tests for Parallel Group Designs in Rare Disease Settings},
  journal = {Statistics in Medicine},
  year = {2026},
  volume = {45},
  number = {20-22},
  doi = {10.1002/sim.70735}
}
Start for free →