Detection of asymptomatic carotid stenosis in patients with lower-extremity arterial disease: development and external validations of a risk score

Abstract Background Recommendations for screening patients with lower-extremity arterial disease (LEAD) to detect asymptomatic carotid stenosis (ACS) are conflicting. Prediction models might identify patients at high risk of ACS, possibly allowing targeted screening to improve preventive therapy and compliance. Methods A systematic search for prediction models for at least 50 per cent ACS in patients with LEAD was conducted. A prediction model in screened patients from the USA with an ankle : brachial pressure index of 0.9 or less was subsequently developed, and assessed for discrimination and calibration. External validation was performed in two independent cohorts, from the UK and the Netherlands. Results After screening 4907 studies, no previously published prediction models were found. For development of a new model, data for 112 117 patients were used, of whom 6354 (5.7 per cent) had at least 50 per cent ACS and 2801 (2.5 per cent) had at least 70 per cent ACS. Age, sex, smoking status, history of hypercholesterolaemia, stroke/transient ischaemic attack, coronary heart disease and measured systolic BP were predictors of ACS. The model discrimination had an area under the receiver operating characteristic (AUROC) curve of 0.71 (95 per cent c.i. 0.71 to 0.72) for at least 50 per cent ACS and 0.73 (0.72 to 0.73) for at least 70 per cent ACS. Screening the 20 per cent of patients at greatest risk detected 12.4 per cent with at least 50 per cent ACS (number needed to screen (NNS) 8] and 5.8 per cent with at least 70 per cent ACS (NNS 17). This yielded 44.2 and 46.9 per cent of patients with at least 50 and 70 per cent ACS respectively. External validation showed reliable discrimination and adequate calibration. Conclusion The present risk score can predict significant ACS in patients with LEAD. This approach may inform targeted screening of high-risk individuals to enhance the detection of ACS.


Introduction
Carotid stenosis is a cause of cerebral infarction in around 15 per cent of ischaemic strokes 1 . Significant asymptomatic carotid stenosis (ACS) is also a predictor of coronary events and vascular death 2,3 , and the risk of both stroke and heart attack can be reduced with appropriate medical therapy. Using duplex ultrasound imaging, the prevalence of moderate or severe (50 per cent or greater) and severe (70 per cent or greater) ACS in the general population has been estimated to be 2.0 and 0.5 per cent respectively 4 . The prevalence is higher in patients with lower-extremity arterial disease (LEAD) 5,6 , but guideline recommendations 7-10 for screening for significant ACS in patients with LEAD vary. Arguments for screening include initiation or improvement of preventive therapies, but also more intensive follow-up to maintain compliance and reduce the risk of subsequent vascular events 11,12 . Arguments against screening include the low prevalence of significant ACS, even among patients with LEAD, and, although medical therapy is indicated in patients with ACS, considerable disagreement exists about the net clinical benefit of routine use of carotid interventions in patients with medically treated ACS.
Risk prediction models allow targeted screening among patients with LEAD at particularly high risk of significant ACS based on multiple risk factors. The aim of this study was to perform a systematic review for published risk prediction models, and further to develop and validate a new risk prediction model for the detection of ACS: the Detection of Asymptomatic Carotid Stenosis in patients with Lower-Extremity Arterial Disease (DACS-LEAD) risk score.

Methods
A systematic review was performed to identify established risk prediction models for the prediction of at least 50 per cent ACS in patients with LEAD. This was conducted according to a predefined protocol that was registered prospectively in the international prospective registry for systematic reviews (PROSPERO) (number CRD42019155482) (Appendix S1).

Derivation cohort
A risk prediction model was developed using data from 3 050 448 self-referred and self-funded individuals who attended commercial vascular screening clinics between 2008 and 2013 in the USA 13,14 . This was referred to as the derivation cohort. All individuals completed a standard questionnaire including questions about their age, sex, height and weight, smoking status, history of hypertension, hypercholesterolaemia, diabetes mellitus and vascular disease (transient ischaemic attack (TIA), stroke and coronary heart disease (CHD)), and use of antiplatelet, antihypertensive and lipid-lowering medication. BP was measured as part of the ankle : brachial pressure index (ABPI) assessment. Standard BP cuffs and sphygmomanometers were used, and systolic BP (SBP) was measured using a Doppler probe.
Most participants underwent carotid duplex screening (conducted by trained staff using dedicated vascular ultrasound instruments: LOGIQ e V R ; GE Healthcare). The highest peak systolic velocity (PSV) and end diastolic velocity (EDV) of both common and internal carotid arteries were measured.
LEAD was defined as an ABPI of 0.9 or less at either side, and these individuals were included in the present analyses. Participants were excluded if they did not undergo ABPI measurement, if ABPI determination was not possible because the arteries could not be compressed (175 517 patients), if ABPI was >0.9 on both sides (2 759 591), if they did not undergo duplex ultrasound imaging (2759) or of they had inconsistent values (464).

External validation cohorts
The risk prediction model was validated by assessing the predictive performance in two independent (external) cohorts. For the first external validation of the risk prediction model, data for 225 691 self-referred and self-funded individuals who attended commercial vascular screening clinics between 2008 and 2013 in the UK were used. The following participants were excluded: those who did not undergo ABPI measurement or in whom ABI was not possible because the arteries could not be compressed (10 774 patients) or in whom ABPI was above 0.9 on both sides (209 276), those who did not undergo duplex ultrasonography (106), or who had inconsistent values (135).
For the second external validation, data from the Second Manifestation of ARTerial disease (SMART) study were used. This is an ongoing prospective cohort at the University Medical Centre Utrecht, the Netherlands. The rationale and design of the SMART study have been published previously 15 . Between September 1996 and October 2019, 13 799 patients with recent diagnosis (1 year before baseline) of a first manifestation of arterial disease, including cerebrovascular disease, coronary artery disease, peripheral arterial disease or aneurysm of the abdominal aorta, were included. After inclusion, individuals completed a questionnaire with questions about medical health and lifestyle, and underwent a standardized vascular screening. Office BP was measured with a non-random sphygmomanometer and the mean of multiple measurements was taken. Use of antithrombotic, BP lowering and cholesterol-lowering medication was recorded. For external validation, baseline characteristics of individuals who had ABPI determination and duplex ultrasound imaging of the carotid arteries were used. Patients with an ABPI above 0.9 on both sides (11 371 patients), in whom no duplex ultrasonography was performed (35), or who had recent (less than 12 months) cerebrovascular symptoms (220) were excluded.

Predicted outcomes
Two outcomes were used. The first, moderate or severe ACS, was defined as at least 50 per cent ACS, based on a PSV of 150 cm/s or more on either side, or 0 cm/s for occluded arteries. The second outcome, severe ACS, was defined as at least 70 per cent ACS; based on a PSV of 210 cm/s or more on either side, or 0 cm/s for occluded arteries.

Model derivation
The development of the prediction model adhered to the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) statement 16 .
Baseline characteristics are presented as mean(s.d.) values for continuous variables and as absolute numbers and percentages for categorical variables. Age was categorized into four groups (<50, 50-59, 60-69 and !70 years), SBP into three groups (<140, 140-159 and !160 mmHg), ABPI into three groups (>0.8 to 0.9, more than >0.4 to 0.8, 0.4), and smoking status was dichotomized as ever smoking versus never smoking. For most predictors, the proportion of individuals with missing data was acceptable (less than 10 per cent), except for smoking status (10.5 per cent), reported history of CHD (13.8 per cent) and stroke/TIA (15.2 per cent) ( Table S1). Missing data were multiple-imputed using chained equations, and 20 data sets with 200 iterations were created. Results were combined with Rubin's rules 17,18 .
The relationship between predictors and the presence of at least 50 per cent and at least 70 per cent ACS in individuals with LEAD was determined with multivariable logistic regression. Predictors were selected based on risk prediction models for significant ACS in the general population 19 , and forward stepwise selection with predictors selected using Akaike information criterion (AIC) 20 . Discrimination and calibration of the developed risk prediction model were examined. Discrimination is the ability of the prediction model to distinguish between patients with and those without the disease outcomes, assessed using the area under the receiver operating characteristic (AUROC) curve. Calibration is the agreement between predicted and observed risk, and was assessed with calibration plots.

Internal validation, score chart and targeted screening
Internal validation was performed to control for potential overfitting. Overfitting quantifies the difference in predictive performance between the data from which the risk prediction model is derived and similar future cases. The predictive performance is expected to be better in the derivation cohort because the logistic model fits the data in an optimal fashion. Bootstrapping to sample the original data set was used and 1000 bootstrap replications per imputed data set were created 21 . The mean calibration slope of the 1000 bootstrap replications in each imputed data set was calculated and used as a uniform shrinkage factor to adjust the regression coefficients for risk of potential overfitting. The shrunken b coefficients were used to calculate the adjusted intercept by fitting a logistic model with the shrunken b coefficients as dependent variables in the original data set. Overoptimism-corrected AUROC curves for each imputed data set were calculated, and the results were combined with Rubin's rules 17,18 .
Regression coefficients of the predictors were converted into points on a score chart to facilitate clinical use of the risk prediction model. One score chart was created for ACS of both 50 and 70 per cent or above. For this, the b coefficients were multiplied by four and then rounded to the closed integer. If the scores for at least 50 and 70 per cent ACS were conflicting, the score for at least 50 per cent ACS was used. The risk of ACS of at least 50 and 70 per cent for the total points (sum scores) was calculated.
Test characteristics of a targeted screening programme using this risk score for selection of participants were calculated. The number needed to screen (NNS) to detect participants with ACS, positive and negative predictive values, sensitivity and specificity for four possible thresholds for targeted screening were calculated.

External validation
External validation assesses the predictive performance of the developed risk prediction model in independent cohorts. Participants who attended commercial vascular screening clinics in the UK and patients from the SMART cohort were used for external validation 15 . The same methods for reporting baseline characteristics and handling missing data were used as in the derivation cohort. The original regression formula (after internal validation) was used to calculate the risk of at least 50 and 70 per cent ACS. Discrimination was assessed using the AUROC curve, and calibration with calibration plots. All predictors could be matched with variables in the validation cohorts, but a proxy was used for history of hypercholesterolaemia in one external validation cohort. The screening approach was based on self-report in the derivation cohort and on blood measurement in the SMART cohort.
Differences between the prevalence of the predicted outcome in the development cohort and the validation cohorts are known to influence calibration. For this reason, the DACS-LEAD was recalibrated to the prevalence of the predicted outcome in the external validation cohorts by adjusting the original intercept. This type of recalibration is referred to as 'update intercept' or 'calibration-in-the-large' 22

Results
Some 4907 unique reports identified by literature searching were screened and 43 full-text reports were assessed for eligibility. No study met the inclusion criteria ( Fig. S1 and Table S2), and no external validation of established risk prediction models could be performed.

Derivation cohort
In total, 112 117 patients with LEAD were used for development. The mean(s.d.) age in the derivation cohort was 70.5(10.7) years and 27.7 per cent were men. Around 50 per cent of patients reported use of aspirin and lipid-lowering therapy, and about 60 per cent reported use of antihypertensive therapy. In patients with significant ACS, around 60 per cent reported use of aspirin and lipid-lowering therapy, and almost 75 per cent reported use of antihypertensive therapy. The overall prevalence of at least 50 per cent ACS in patients with LEAD was 5.7 per cent and that for at least 70 per cent ACS was 2.5 per cent ( Table 1).

Risk prediction model development and internal validation
The following predictors were included: age, sex, ever smoking versus never smoking, history of hypercholesterolaemia, stroke/ TIA, CHD, measured SBP, and ABPI. The AUROC curve adjusted for overoptimism was 0.71 (95 per cent c.i. 0.71 to 0.72) for at least 50 per cent ACS and 0.73 (0.72 to 0.73) for at least 70 per cent ACS ( Table 2). Internal validation with bootstrapping techniques indicated that no shrinkage of the b coefficients was needed. Calibration plots showed good concordance between predicted and observed risk for ACS of both at least 50 and 70 per cent, indicating that groups of patients at both low and high risk can be predicted reliably by the DACS-LEAD risk score (Fig. 1a,d).

Targeted screening to detect asymptomatic carotid stenosis
The DACS-LEAD score chart is provided in Table 3. Sum scores ranged from 0 to 22. The prevalence of at least 50 per cent ACS ranged from 0.5 per cent for sum scores of 7 or less to 14 per cent  for sum scores of 16 or more. The prevalence of at least 70 per cent ACS ranged from 0.2 per cent for sum scores of 7 or less to 6.6 per cent for sum scores of 16 or more. The prevalence of at least 50 and 70 per cent ACS by each sum score is shown in Fig. 2. The distribution of sum scores is shown in Fig. S2. Four thresholds of sum scores were introduced, allowing targeted screening of a group of patients with LEAD at high risk of significant ACS. The observed prevalence of at least 50 per cent ACS increased from 5.7 per cent by screening all patients to 12.4 per cent by targeted screening of the 20 per cent of patients at very high risk. The corresponding NNS decreased from 18 to 8. The observed prevalence of at least 70 per cent ACS increased from 2.5 per cent by screening all patients to 5.8 per cent by targeted screening of the 20 per cent of patients at very high risk. The corresponding NNS decreased from 40 to 17 (Table 4).

External validation populations
In the first validation cohort (Life Line Screening; UK participants), compared to the derivation cohort more patients were men (37.2 per cent) and current smokers (32.2 per cent), and fewer patients had diabetes mellitus (16.7 per cent) and  Table S3. The DACS-LEAD score ranged from 0 to 22. Risks of at least 50 and 70 per cent asymptomatic carotid stenosis for each sum score are shown in Fig. 2. TIA, transient ischaemic attack; CHD. coronary heart disease; SBP, systolic BP; ABPI, ankle : brachial pressure index. hypercholesterolaemia (44.8 per cent) ( Table 1). The prevalence of at least 50 per cent ACS was 9.1 per cent and that for at least 70 per cent ACS was 4.8 per cent. In the second validation cohort (SMART), patients were younger (mean(s.d.) age 61.6(10.3) years) and more were men (66.5 per cent) than in the derivation cohort.
The prevalence of at least 50 per cent ACS was 16.8 per cent and that for at least 70 per cent ACS was 12.4 per cent.

External validation
The  Table 2). Risks were underestimated when the risk score was tested in the validation cohorts (Fig. S3), but calibration plots showed good concordance between the predicted prevalence calculated with the DACS-LEAD and the observed prevalence in the validation cohorts (after adjusting the intercept) (Fig. 1). This indicates that, after adjusting the mean predicted risk to the observed risk in the validation cohorts, the DACS-LEAD risk score could be applied to populations with LEAD with different overall prevalence of significant ACS.

Discussion
Predictors for moderate and severe ACS used in the DACS-LEAD risk score were age, sex, ever smoking, history of hypercholesterolaemia, stroke/TIA, CHD, and measured SBP and ABPI. Discrimination was good, and calibration plots showed good concordance of predicted and observed risks. The NNS to detect significant ACS in the high-risk group that consisted of approximately 20 per cent patients at highest risk was more than halved compared with systematic screening of all patients with LEAD. Application of DACS-LEAD to the validation cohort showed reliable predictions (after adjusting for the difference in overall prevalence of significant ACS in the different validation cohorts). The DACS-LEAD risk score can be applied to patients with decreased ABPI of 0.9 or below. These ABPI values have been associated with cardiovascular morbidity and mortality, and might also improve cardiovascular risk stratification 23,24 . The prevalence of significant ACS is higher in patients with LEAD and increases with the severity of LEAD 25 . A meta-analysis 6 including 13 prospective studies of patients with LEAD showed a prevalence of at least 50 per cent ACS of 25 per cent, and a prevalence of at least 70 per cent ACS of 14 per cent, but heterogeneity between studies was high owing to different selection  criteria and the application of different diagnostic criteria to determine the degree of carotid stenosis. The prevalence in the validation cohorts was higher than in the derivation cohort due to the inclusion of patients with a first manifestation of arterial disease in the SMART cohort, and possibly different reasons for undergoing vascular screening in the USA and the UK. This hampers direct implementation of the present risk score in clinical practice. Adjustment of the prediction model (while keeping the included predictors) to new settings is necessary. Another approach might be to develop different risk models based on the disease severity of LEAD. The Fontaine or Rutherford classifications could be used for patient selection, and the risks of ACS calibrated accordingly (for example, separate predictions for Fontaine stage IIb-IV versus I-IIa) to provide more consistent predictions of ACS across cohorts. Detection of significant ACS in patients with asymptomatic LEAD might lead to the initiation or intensification of medical therapy. In patients with symptomatic LEAD who were receiving optimal medical therapy, detection might require more intensive follow-up to maintain compliance 11,12 . Compliance with medical therapy is challenging, as recently shown in a study from Sweden 25 . In this study of screening for ACS in men aged 65 years, statins and antiplatelet agents were prescribed when ACS was detected, but were used by 29 and 21 per cent of patients respectively at 5 years after detection, compared with approximately 23 and 14 who used them at age 65 years 26 . Furthermore, intensification of both lipid-lowering therapy 27,28 , antiplatelet therapy 29 , and a combination of low-dose aspirin and low-dose rivaroxaban might be particularly beneficial in terms of stroke prevention in patients with significant ACS 30 .
Although the annual risk of ipsilateral stroke in patients with significant ACS using best medical therapy is low 31,32 , the 5-10-year risk may not be negligible, especially in patients with severe ACS. 33 Risk factors for increased stroke risk in medically treated patients have been identified, but risk models have not been validated in patients using current evidence-based medical therapy 34,35 . Carotid interventions might be considered in selected patients to reduce stroke risk further, but the benefit of carotid interventions should, amongst other considerations, be weighed against the limited life expectancy in patients with LEAD 9 .
Participants of the derivation cohort and one of the validation cohorts were self-referred and self-funded, possibly influencing generalizability to other populations. The derivation cohort was not designed primarily for research purposes, but participants were identified prospectively. ABPI and PSV as single measurements for diagnosing LEAD and ACS may be useful as a screening tool to identify patients for more intensive diagnostic work-up. Substantial differences in the overall prevalence of ACS across cohorts were found. The timing of stroke/TIA was not recorded, and some patients may have had recent cerebrovascular symptoms from their carotid stenosis. For predictors that were selfreported, recall bias should be taken into account. BP was measured once in the US and UK cohorts, and might not reflect 'usual' values. A proxy for history of hypercholesterolaemia was used in the SMART cohort, which might have influenced external validity. The prevalence of ACS in the derivation cohort was lower than in other populations, possibly making targeted screening more worthwhile in different settings 6 .
The present study has several strengths. The first risk score to detect significant ACS in individuals with LEAD was developed and validated in two populations. A large cohort of individuals was used for development of the DACS-LEAD risk score. Missing data were limited for most predictors in the derivation and validation cohorts, and multiple imputation was used to handle missing data. Internal validation showed no evidence of overfitting, and external validation showed reliable prediction after recalibration. ABPI was measured bilaterally, and individuals with incompressible ankle arteries or an ABPI above 1.4 were excluded. Individuals underwent bilateral examination of the carotid arteries, and the highest degree of stenosis of both sides was used as outcome.
Before targeted screening can be implemented in clinical practice, future research will determine risks of cardiovascular events in individuals with LEAD and concomitant significant ACS under best medical therapy, will identify patients with significant ACS at increased risk of ischaemic stroke who benefit from carotid interventions, and whether closer follow-up and better compliance after detecting significant ACS might improve cardiovascular risk management and prevent cardiovascular events.
The DACS-LEAD risk score can provide individualized predictions of the presence of moderate and severe ACS in individuals with LEAD, using the following predictors: age, sex, smoking status, history of hypercholesterolaemia, stroke/TIA, CHD, and measured SBP. Using the risk score to select individuals for a targeted screening programme, approximately 40 per cent of patients with ACS were detected by screening only 20 per cent of individual participants. The yield increased to approximately 70 per cent when 40 per cent of participants were screened. The NNS was reduced substantially, and the present risk score could help to target screening in those whom ACS is more likely to be detected.