-
Views
-
Cite
Cite
Vladimir Vovk, Ruodu Wang, Combining p-values via averaging, Biometrika, Volume 107, Issue 4, December 2020, Pages 791–808, https://doi.org/10.1093/biomet/asaa027
- Share Icon Share
Summary
This paper proposes general methods for the problem of multiple testing of a single hypothesis, with a standard goal of combining a number of |$p$|-values without making any assumptions about their dependence structure. A result by Rüschendorf (1982) and, independently, Meng (1993) implies that the |$p$|-values can be combined by scaling up their arithmetic mean by a factor of 2, and no smaller factor is sufficient in general. A similar result by Mattner about the geometric mean replaces 2 by e. Based on more recent developments in mathematical finance, specifically, robust risk aggregation techniques, we extend these results to generalized means; in particular, we show that |$K$| |$p$|-values can be combined by scaling up their harmonic mean by a factor of |$\log K$| asymptotically as |$K$| tends to infinity. This leads to a generalized version of the Bonferroni–Holm procedure. We also explore methods using weighted averages of |$p$|-values. Finally, we discuss the efficiency of various methods of combining |$p$|-values and how to choose a suitable method in light of data and prior information.