U-PASS: unified power analysis and forensics for qualitative traits in genetic association studies

Abstract

Summary

Despite the availability of existing calculators for statistical power analysis in genetic association studies, there has not been a model-invariant and test-independent tool that allows for both planning of prospective studies and systematic review of reported findings. In this work, we develop a web-based application U-PASS (Unified Power analysis of ASsociation Studies), implementing a unified framework for the analysis of common association tests for binary qualitative traits. The application quantifies the shared asymptotic power limits of the common association tests, and visualizes the fundamental statistical trade-off between risk allele frequency and odds ratio. The application also addresses the applicability of asymptotics-based power calculations in finite samples, and provides guidelines for single-SNP-based association tests. In addition to designing prospective studies, U-PASS enables researchers to retrospectively assess the statistical validity of previously reported associations.

Availability and implementation

U-PASS is an open-source R Shiny application. A live instance is hosted at https://power.stat.lsa.umich.edu. Source is available on https://github.com/Pill-GZ/U-PASS.

Supplementary information

Supplementary data are available at Bioinformatics online.

1 Introduction

The probability of detecting a true association between genetic and phenotype variations, known as statistical power, is influenced by a number of factors such as sample sizes, the statistical test used, the frequency of the risk variant and magnitude of the effect on the trait. Power analysis, which determines the suitable factor combinations in order to achieve sufficient statistical power, plays an important role in determining study designs (Goodwin et al., 2016; Skol et al., 2006), and in interpreting published findings (Ioannidis, 2005).

There has been a number of widely used calculators for genome-wide association studies (GWAS). Sham (1998) studied power analysis of likelihood ratio tests for associations between marker SNPs and quantitative or qualitative traits; the results were implemented in GPC (Purcell et al., 2003). Skol et al. (2006) studied the performance of two-sample t-tests, and extended the analysis to two-stage designs; the results were implemented in the CaTS calculator, and later, in the GAS calculator for one-stage designs (Johnson and Abecasis, 2017). Independently, Menashe et al. (2008) implemented the calculations for one-stage designs in the PGA calculator. Recent works have also studied power of a number of SNP-set-based tests targeting rare variants (Derkach et al., 2018; Wang et al., 2014). See Sham and Purcell (2014) for a review.

Despite these efforts, some difficulties remain in practice:

1. Lack of universality. Existing power analyses are tied to the underlying models and the statistical procedures used; power calculations for a certain model-method combination may not be valid if either the model or the method changes. Users are burdened with matching the appropriate tool to the specific type of analysis they wish to perform. This is complicated by the fact that the precise test and model assumptions are rarely made explicit in the existing calculators.

2. Mismatching definitions of key quantities. While GWAS catalogs, e.g. NHGRI-EBI (MacArthur et al., 2017), require studies to report risk allele frequency (RAF) in the control group, all of the aforementioned power calculators assume the RAF input to be the frequency in the general population. These quantities are not necessarily equal, and using one in place of the other may grossly distort power estimates.

3. (In)accuracies in finite samples. While existing tools rely on large-sample approximations in their power calculations, these approximations are not reliable in finite samples when genetic variants are rare. Existing calculators are silent about the applicability of asymptotics-based approximations, and how they should be corrected.

As a result, it is not only challenging to use the existing power calculation tools for planning genetic association studies correctly, but also difficult to systematically review the statistical validity of findings reported in the literature, since different models and tests must be handled differently, and with care.

In an effort to address these difficulties and deficiencies, we propose a unified framework for power analysis of single variant association studies. By abstracting away the assumptions of disease models and testing procedures which may vary from study to study, we reduce the problem to the essential quantities that are invariant to nuisance parameters. These ideas are implemented in the software U-PASS (Unified Power analysis of ASsociation Studies), enabling model-invariant, test-independent power analysis as well as systematic reviews of the statistical validity of reported findings.

We briefly summarize the important features and uses of the software below. Mathematical details and results from numerical experiments are collected in the Supplementary Material.

2 Methods and features

2.1 The canonical and disease model parametrizations

We provide two ways of specifying the alternative hypothesis in power analysis. In addition to specification through disease models, we provide users with the option to perform power analysis by specifying the canonical parameters, whose estimates are reported and curated in the NHGRI-EBI GWAS Catalog:

• Conditional distribution of risk allele variant among controls, i.e. RAF in the Control group, denoted as f.

• Odds ratio (OR) of allele variants, denoted as R.

The canonical parameters (f and R) are common to models of qualitative traits and invariant to model choices. This disease model-invariance allows users to perform power analysis valid for studies employing different models. We also elucidate on the link between the two approaches, and provide a ‘disease model converter’ in the application, performing explicit conversions from the disease models to the canonical parametrizations.

2.2 A test-independent power analysis

While power calculations are necessarily tied to the statistical tests used, for many common association tests, statistical powers are asymptotically equivalent. We show in the Supplementary Material that the likelihood ratio test, χ² test, Welch’s t-test and LR test for logistic regressions have asymptotically the same power, as long as the canonical parameters assume the same values. This test-independent analysis allows us to calculate power in a unified fashion, regardless of the statistical tests used. In particular, when performing retrospective analysis, users need only specify the number of cases and controls. The common power limits are calculated as a function of RAF and OR, and visualized as a heatmap in the OR-RAF diagram. The formulas used for power calculations in terms of the canonical parameters are detailed in Supplementary Material Section 4.

2.3 Review and forensics of reported findings

This unified treatment allows us to examine results from different studies in the same diagram, even when they do not employ the same model or statistical test. This enables systematic reviews of reported findings for their statistical validity. In particular, a reported association predicted to have low power given the study’s sample size—lying in the dark regions of the OR-RAF diagram—while not impossible, invites further scrutiny.

Studies where reported associations show misalignment with the predicted equi-power curves may be further investigated for potential problems. We reached out to one study where gross misalignment was identified (Domínguez-Cruz et al., 2018). The authors of the study confirmed that this was the result of a problem in the data curation process of the GWAS Catalog (Domínguez-Cruz, personal communication).

The software provides options for users to load and overlay findings reported in the NHGRI-EBI GWAS Catalog, or upload data from other sources compliant with the Catalog’s data format.

2.4 Rare variants and finite sample corrections

We address the quality of asymptotic approximations in our power analysis, as well as the applicability of single variant tests when rare genetic variants are present. Specifically, we provide a lower bound on the variant counts needed to calibrate Fisher’s exact test. If variant counts fall below the threshold, exact tests, and by extension, single-SNP-based association tests, cannot be correctly calibrated to have the desired Type I error rate without sacrificing all statistical power. In such cases, the asymptotic approximations do not apply. We mark this low-count, low-power region on the OR-RAF diagram. See Supplementary Material Section 5 for further details.

The software also provides options for users to specify the rare variant as (i) having less than a specified count or (ii) occurring in less than a percentage of all subjects in the study, as is customary in the literature.

3 Implementation

U-PASS is implemented as an interactive web-based R Shiny application, hosted at https://power.stat.lsa.umich.edu, open to the public. Source code is freely available at https://github.com/Pill-GZ/U-PASS.

Funding

This work is partially supported by NSF Grant DMS-1830293, ATD.

Conflict of Interest: C.V.V.H. is an employee at Regeneron Genetics Center.