Sampling Strategies for Internal Validation Samples for Exposure Measurement–Error Correction: A Study of Visceral Adipose Tissue Measures Replaced by Waist Circumference Measures

Abstract Statistical correction for measurement error in epidemiologic studies is possible, provided that information about the measurement error model and its parameters are available. Such information is commonly obtained from a randomly sampled internal validation sample. It is however unknown whether randomly sampling the internal validation sample is the optimal sampling strategy. We conducted a simulation study to investigate various internal validation sampling strategies in conjunction with regression calibration. Our simulation study showed that for an internal validation study sample of 40% of the main study’s sample size, stratified random and extremes sampling had a small efficiency gain over random sampling (10% and 12% decrease on average over all scenarios, respectively). The efficiency gain was more pronounced in smaller validation samples of 10% of the main study’s sample size (i.e., a 31% and 36% decrease on average over all scenarios, for stratified random and extremes sampling, respectively). To mitigate the bias due to measurement error in epidemiologic studies, small efficiency gains can be achieved for internal validation sampling strategies other than random, but only when measurement error is nondifferential. For regression calibration, the gain in efficiency is, however, at the cost of a higher percentage bias and lower coverage.

Throughout the main paper, our interest is the causal effect of the exposure VAT on the outcome insulin resistance IR, adjusted for a predefined set of k confounders, jointly written as Z (e.g., age, sex and total body fat). We assume a linear model for the outcome without interaction between exposure and covariates: Here, we assume that the residuals errors ε are independent of VAT and confounders Z, with mean 0 and variance σ 2 . Additionally, γ is assumed a k × 1 vector of regression coefficients. The parameter β in equation (W1) is the parameter of interest. We consider the setting that instead of the exposure of interest, VAT, WC is measured. The variable WC is the error-prone substitute measure for VAT, where we assume that WC = θ 1 VAT + U , where U is a random variable, with mean 0 and variance τ 2 , and U is assumed independent of VAT. The factor θ 1 is a scalar, used to scale VAT to the same scale as WC. We also assume non-differential measurement error, i.e., WC |= Y |VAT, Z. This form of measurement error is referred to as random (or sometimes classical) measurement error if θ = 1 and systematic (or sometimes linear) measurement error otherwise [1,2]. Since the substitute measure is often measured on a different scale than the true measure, measurement error will often be of the systematic form. Using WC instead of VAT in the linear model yields: E[IR|WC, Z] = intercept * + β * WC + γ * Z.
Under this model, by the law of total expectation, we have E[IR|WC, Z] = intercept + β × E[VAT|WC, Z] + γ * Z, which relies on the assumption that the measurement error is non-differential [3]. It follows that, β * = αβ with α = Cov(WC, VAT|Z) Var(WC|Z) = θ 1 Var(VAT|Z) θ 2 1 Var(VAT|Z) + U . (W3) In conclusion, the ordinary least squared estimator for β * is biased for β by a factor α. This factor is sometimes referred to as the attenuation factor in case of random measurement error, since Var(VAT|Z) < Var(VAT|Z) + U and hence, α < 1.

The different analyses with internal validation samples
When a study contains an internal validation sample for which information is available on both WC and VAT, different analyses can be conducted. Five different estimators are explained below. The variance of these estimators can be obtained from standard output of statistical software when no further details on variance estimation are provided below. The internal validation sample restricted analysis relies on the assumption that the VAT measures in the main study are completely missing at random and the regression calibration methods rely on the assumption that measurement error in WC is non-differential.
Uncorrected analysis. The measurement error is ignored and the relation between VAT and IR is estimated using the error-prone substitute measure WC. Under the assumptions above, as shown in equation W3, this estimator is biased by a factor α.
Internal validation sample restricted analysis. The association between VAT and IR is determined using only the data from the internal validation sample (in which a direct measure of VAT is available). This approach will naturally yield unbiased estimates if measures of VAT are missing completely at random in the main study, but power of the study will substantially decrease as only a part of the data available in the main study is used.
Standard regression calibration. The basis of regression calibration is the replacement of WC by a corrected version of WC, based on the regression of VAT on WC and the confounders Z. In this way, the induced measurement error in the uncorrected analysis is corrected by regressing the outcome IR on the confounders Z and E[VAT|WC, Z] instead of WC (i.e., by using the predicted values from regressing VAT on WC and Z, instead of WC). This method is identical to dividing the least squares estimator β * in equation W2 by the correction factor α defined in equation W3 [2]. The variance of this estimator can be estimated by applying the Delta method described by Rosner et al. [4].
Efficient regression calibration. This analysis pools the estimator of the internal validation sample restricted analysis with the regression calibration estimator, by using weights equal to the inverse of the variance of the two estimates, and was described by Spiegelman et al. [5]. This approach is called efficient regression calibration since it makes use of the fact that in the individuals included in the internal validation sample, VAT is actually known and does not neglect this information. The variance of this estimator can be estimated by taking the inverse of: the sum of the inverse of the variance of the internal validation sample restricted estimator and the inverse of the variance of the regression calibration estimator, as described by Spiegelman et al. [5].
Validation regression calibration. This analysis uses the predicted values from regressing VAT on WC and Z for individuals outside the internal validation sample and VAT otherwise. We call this approach validation regression calibration approach since this is the standard regression calibration approach in internal validation studies [1]. Validation regression calibration treats the predicted values as if they were known and therefore neglects their uncertainty.

Web Appendix 2
In the simulation study presented in the main text, the measurement error variance τ and the parameter λ in the gamma distribution of the residual errors of VAT were varied according to the R-squared of the measurement error model and skewness of the residuals errors, respectively. The corresponding values for τ and λ in the data generating mechanism found in the main text can be found in Web Table 1.
Web Table 1. Values of the parameters R-squared and skewness varied in the simulation study in a full factorial design. The values for τ and λ present the values for that parameter in the data generating mechanism that corresponds to the given R-squared and skewness, respectively.

Web Appendix 3
The results of the simulation study that were left out the main text for brevity are shown in the following subsections. Full results of the simulation study can also be found at an online repository [6]. Specifically, R compatible summary files are available at www. github.com/LindaNab/me_neo/results/summaries. These summary files contain more detailed information on e.g. model based standard errors, empirical standard errors and Monte Carlo standard errors. Additionally, output of each single run of the simulation study can be found at www.github.com/LindaNab/me_neo/data/output and subsequent folders.

Internal validation restricted analysis
The main results of the internal validation restricted analysis were shown in the main text. Web Figure 1 shows the mean squared error of the association between visceral adipose tissue and insulin resistance using an internal validation sample of 25% of the main study's sample size. Web Table 2 shows the mean squared error of the association under study in the scenarios where R-squared was equal to 0.9 or skewness was equal to 1.0, that were left out the main text for brevity. Web Table 3 shows the percentage bias and coverage of the association under study in the scenarios where R-squared was equal to 0.9 or skewness was equal to 1.0. Web Table 2. Mean squared error of the estimated association between visceral adipose tissue and insulin resistance in the analysis restricted to the internal validation sample   Table 3. Percentage bias and coverage of the estimated association between visceral adipose tissue and insulin resistance in the analysis restricted to the internal validation sample Scenario IVS a 40% of main study IVS a 25% of main study IVS a 10% of main study Linear Skewness

Validation regression calibration
The main results of validation regression calibration were shown in the main text. Web Figure 2 shows the mean squared error of the association between visceral adipose tissue and insulin resistance using an internal validation sample of 25% of the main study's sample size. Web Table 4 shows the mean squared error of the association under study in the scenarios where R-squared was equal to 0.9 or skewness was equal to 1.0, that were left out the main text for brevity. Web Table 5 shows the percentage bias and coverage of the association under study in the scenarios where R-squared was equal to 0.9 or skewness was equal to 1.0. Web Figure 2. Nested loop plot of the mean squared errors in the analysis using validation regression calibration to correct for the measurement error for the three different sampling strategies. A) Linear measurement error model and an internal validation sample of 25% of the main study; and B) Non-linear measurement error model and an internal validation sample of 25% of the main study. Order from outer to inner loops: Skewness of the residual errors of the gold standard measure (S, 3 levels, increasing); R-squared of the measurement error model (R 2 , 4 levels, increasing).
Web Table 4. Mean squared error of the estimated association between visceral adipose tissue and insulin resistance in the validation regression calibration analysis Scenario IVS a 40% of main study IVS a 25% of main study IVS a 10% of main study

Efficient regression calibration
The results of the application of efficient regression calibration for measurement error correction were as follows. Web Figure 3 shows the mean squared error of the association between visceral adipose tissue and insulin resistance using an internal validation sample of 10%, or 40% of the main study's sample size. Web Figure 4 shows the mean squared error of the association between visceral adipose tissue and insulin resistance using an internal validation sample of 25% of the main study's sample size. Web Table 6 shows the mean squared error of the association under study in the scenarios where R-squared was equal to 0.9 or skewness was equal to 1.0, that were left out Web Figure 3 and 4 for comparability with Figure 5-6 in the main text. Web Table 7 shows the percentage bias and coverage of the association between visceral adipose tissue and insulin resistance using an internal validation sample of 10%, 25% or 40% of the main study's sample size.
Web Table 6. Mean squared error of the estimated association between visceral adipose tissue and insulin resistance in the efficient regression calibration analysis   Web Figure 4. Nested loop plot of the mean squared errors in the analysis using efficient regression calibration to correct for the measurement error for the three different sampling strategies. A) Linear measurement error model and an internal validation sample of 25% of the main study; and B) Non-linear measurement error model and an internal validation sample of 25% of the main study. Order from outer to inner loops: Skewness of the residual errors of the gold standard measure (S, 3 levels, increasing); R-squared of the measurement error model (R 2 , 4 levels, increasing).

Web Table 7. Percentage bias and coverage of the estimated association between visceral adipose tissue and insulin resistance by application of efficient regression calibration
Scenario IVS a 40% of main study IVS a 25% of main study IVS a 10% of main study Linear Skewness

Standard regression calibration
The results of the application of standard regression calibration for measurement error correction were as follows. Web Table 8 shows the mean squared error of the association between visceral adipose tissue and insulin resistance using an internal validation sample of 10%, 25% or 40% of the main study's sample size. Web Table 9 shows the percentage bias and coverage of the association between visceral adipose tissue and insulin resistance using an internal validation sample of 10%, 25% or 40% of the main study's sample size.
Web Table 8. Mean squared error of the estimated association between visceral adipose tissue and insulin resistance in the standard regression calibration analysis