Concordance for prognostic models with competing risks

The concordance probability is a widely used measure to assess discrimination of prognostic models with binary and survival endpoints. We formally define the concordance probability for a prognostic model of the absolute risk of an event of interest in the presence of competing risks and relate it to recently proposed time-dependent area under the receiver operating characteristic curve measures. For right-censored data, we investigate inverse probability of censoring weighted (IPCW) estimates of a truncated concordance index based on a working model for the censoring distribution. We demonstrate consistency and asymptotic normality of the IPCW estimate if the working model is correctly specified and derive an explicit formula for the asymptotic variance under independent censoring. The small sample properties of the estimator are assessed in a simulation study also against misspecification of the working model. We further illustrate the methods by computing the concordance probability for a prognostic model of coronary heart disease (CHD) events in the presence of the competing risk of non-CHD death.


A. Software implementation
The presented concordance estimators for competing risks have been implemented in the function cindex of the R package pec (Mogensen and others, 2012). The following box shows R code for evaluatingĈ 1 (t) at t = 1, 5, and 10 simultaneously for a Fine-Gray regression model and the combination of cause-specific Cox regression with age as the only covariate based on a marginal Kaplan-Meier model for the censoring distribution.
We first generated n = 10, 000 independent marker values X by repeatedly drawing from the standard normal distribution.

(B.1)
This was implemented by simulating latent exponentially distributed event times T 1 and T 2 and then setting T = min(T 1 , T 2 ) and D = 1 for T 1 < T 2 and D = 2 for T 1 T 2 .
We further set λ 01 = 1 and consider three values for λ 02 = {0, 0.5, 2} where λ 02 = 0 corresponds to no competing risks. In each scenario, we compute the untruncated concordance indexC 1 in a single data set of size n = 10, 000 obtained for varying values of β 1 between 0 and 10. Results are shown in Figure 1.
A comparison of scenario (1) between the panels of figure 1 (black solid lines) shows only small changes in concordance index due to the presence of a competing risk. In scenario (2) where the covariate is associated with an increased hazard of the event of interest but a decreased hazard of the competing event, the discrimination ability is increased (black dashed lines). In contrast, the discrimination ability is markedly reduced when the marker affects both cause-specific hazards with regression coefficients of the same sign (scenarios (3) and (4); gray solid and dashed lines).
These behaviors of the concordance probability for the event of interest can be explained by the fact that the overall effect of a covariate on the cumulative incidence function of the Fig. 1. Concordance for cause-specific hazards models depending on the effect size β1 as well as on the baseline hazard λ02(t) and the regression coefficient for the competing risk. Black solid lines refer to β2 = 0, black dashed lines to β2 = −β1, gray solid lines to β2 = β1, and gray dashed lines to β2 = 2β1. Concordance C 1 λ 2 (t) = 2 event of interest depends on both cause-specific baseline hazards and both cause-specific hazard ratios (Beyersmann and others, 2007;Koller and others, 2012). In particular, a covariate that is positively associated with the cause-specific hazard of the event of interest can simultaneously be negatively associated with the corresponding cumulative incidence function of that event if the cause-specific hazard of the competing event is larger or if it shows a stronger positive association with the covariate. This explains that the concordance index drops below 0.5 in scenario (4).
Importantly, these findings suggest that the discrimination accuracy of a prognostic model for the absolute risk of the event of interest improves when there are risk factors for the event of interest which are only weakly or, even better, reversely associated with the cause-specific hazard of the competing event. Control definition 1 considers those as controls who are at risk beyond time s using a standard risk set definition whereas for definition 2, subjects experiencing competing event remain evaluable as controls indefinitely (Fine and Gray, 1999). To motivate our definition of concordance in Section 2.1, we looked at the example of evaluating the benefit of a specific treatment for the event of interest which does not affect the competing event and argued that (cumulative and incident) cases would have a more immediate need for treatment than controls according to definition 2.
In particular, subjects experiencing a competing event have no benefit from treatment at all. In this situation, the control definition 2 would be most relevant. However, in other situations, a treatment might affect both event types, and it would be most relevant to distinguish cases from those who haven't had any event up to that time point, i.e. to use control definition 1.
For incident cases and the second definition of controls (I2), the AUC for the event of interest is defined as with dF 1 (s|X) = f 1 (s|X)ds. By comparing the above formula with (2.2) in the main text, it is easy to see that the concordanceC 1 is a weighted average of the AU C 1,I2 (s) over time: This is in analogy with a similar result for survival analyis (Heagerty and Zheng, 2005).
Similarly, (Zheng and others, 2012;Blanche and others, 2013) defined an AU C(s) for cumulative cases and definition 2 of controls (C2): AU C(s) definitions for control definition 1 have also been proposed. For example, the AU C(s) for cumulative cases and definition 1 of controls (C1) is given by: Of note, accuracy measures using control definition 1 also depend on the cumulative distribution function of the competing event F 2 (s|X). Thus, such measures might be less suitable if the main goal is to assess the relevance of a marker (or a prognostic model) for predicting the absolute risk of the event of interest only but could be valuable for assessing joint models for the cumulative incidence of both competing events.
D. Proof of Lemma 3.1 and estimation of the asymptotic variance

D.1 Consistency
It is assumed thatĜ is consistent for G. Thus, Slutsky's lemma shows that the weights converge in probability to as n → ∞. By the law of large numbers and Slutsky's Lemma it follows thatĈ 1 (t) converges in probability to By applying equations (3.1) and (3.2) we have and hence Similarly by equation (3.2) we have Inserting shows that expression (D.1) equals the expression for C 1 (t) given in (2.4): . 8

D.2 Weak convergence
If the estimate of the conditional censoring distribution converges weakly, more precisely under the following assumption: then the weak convergence ofĈ 1 (t) follows from the i.i.d. representation where IF C1 is the influence function ofĈ 1 which depends on IF G . A rigorous proof of the i. In what follows we assume independent censoring and denote G(t) for the probability of being uncensored at time t, that is G(t) = P(C > t), and denote ST (t) = P(T > t). In addition, we respectively denote by the shorthand notations C num 1 (t) and C den 1 (t) the numerator and the denominator of C 1 (t) as defined at equation (2.4).
The influence function of the marginal Kaplan-Meier estimator for the marginal censoring survival function is given by is the cumulative hazard function of the censoring variable (see for instance Gill, 1994).
For allT i < τ andT j < τ , combining first order Taylor expansions of the function (x, y) → 1/(xy) at (x, y) = Ĝ (T i −),Ĝ(T i ) and at Ĝ (T i −),Ĝ(T j −) with (D.2) and (D.4) yields when defining Note thatŴ −1 ij,1 and so its influence function does no longer depend on observation from subject j when assuming the censoring independent of the covariates. In addition, let us respectively denote the numerator and denominator ofĈ 1 (t) defined at equation (3.4) divided by n 2 byĈ num 1 (t) andĈ den 1 (t). As a consequence of (D.5),(D.6),(D.7), (D.8) and of the expression of the numerator of (3.4) it follows By the same arguments, a similar result holds for √ n Ĉ den 1 (t) − C den 1 (t) : the decomposition is the same than the one of the right term of equation (D.9) except that C num 1 (t) is replaced by C den 1 (t) and that the term Q ij disappears. For all t < τ , a first order Taylor expansion of (x, y) → x/y at (x, y) = Ĉ num 1 (t),Ĉ den 1 (t) further leads to: Finally, we use the Hájek projection principle and U-statistic theory (Van der Vaart, 1998, Sec. 11.3 & Chap. 12) to write the i.i.d decomposition (D.3). More precisely, we first define the symmetric kernel h ijk (t) = ψ ijk (t) + ψ jik (t) + ψ jki (t) /3! following page 171 of Serfling (1980) before applying Theorem 12.3 of Van der Vaart (1998) to it. It therefore follows: D.3 Consistent estimation of the standard error ofĈ 1 Let us denoteM C k (t) = I{T k t,D k = 0} − t 0 I{T k u}dΛ C (u) withΛ C (·) the usual Nelson-Aalen estimator of the cumulative hazard function of the censoring variable C, and letŜT (·) be the empirical estimator of ST (·). By defining a plug-in estimateψ ijk (t) and using empirical means for conditional expectation estimation, we can consistently estimate IF C1 (t;T i ,D i , X i ) by A consistent estimator of the asymptotic variance ofĈ 1 (t) is therefore defined bŷ Table 1. Average bias and root mean square error (RMSE) for 3 different estimators of C1(t) averaged over 1000 data sets simulated under the 2 scenarios CR1 and CR2 for varying sample size N , independent (γ1 = 0) or covariate dependent censoring (γ1 = 1), respectively, and varying censoring rates. t was chosen as the median of the marginal time-to-event distribution. Column 3 shows the expected proportion of right-censored event times amongst observations withT < t. Columns 4-6 show average bias (RMSE) for the three estimators (multiplied by 100 for easier readability).