Response: Re: Comparative Effectiveness of Prostate Cancer Treatments: Evaluating Statistical Adjustments for Confounding in Observational Data

Extract

We are grateful to Dr Vickers for pointing out our careless error in reporting the hazard ratio from our study; we inverted the reference group, which resulted in our overstating the extent of the similarity between the hazard ratio estimated using the instrumental variable method and the results from the benchmark randomized controlled trial. However, although this error is embarrassing, we do not believe that it fundamentally alters our conclusion about the potential strengths and weaknesses of instrumental variable estimation and propensity score reweighting in this particular analysis.

We recognize that instrumental variable analysis is often criticized for generating estimates with large SEs, essentially because the instrument reduces the amount of observed variation in the treatment ( 1 ). Nonetheless, in the absence of well-designed and executed randomized clinical trials, researchers must address the dual problems of bias and imprecision inherent in using observational data to estimate treatment effects. Given the potentially large, but essentially unknowable, influence of unobserved factors on treatment choices and outcomes in observational data, analysts inevitably have to weigh the trade-offs between a potentially biased but precise propensity score–reweighted estimate and a less precise but consistent (unbiased in large samples) instrumental variable estimate. Our propensity score–reweighted hazard ratio estimates, which are very precise, are very similar in value to the unadjusted and equally precise observational estimates [tables 4 and 5 in ( 2 )]. Moreover, crude relative risks calculated from the mortality rates in table 2 ( 2 ) strongly suggest that the rates based on partitioning the sample into two groups according to the median value of the instrument are closer in value to the randomized clinical trial results than are either the observational or propensity score–reweighted relative risks. These calculations are not subject to the large SEs obtained from using the instrumental variable approach to estimate the proportional hazard models.