Sex-specific prediction of cardiogenic shock after acute coronary syndromes: the SEX-SHOCK score

Abstract

Background and Aims

Cardiogenic shock (CS) remains the primary cause of in-hospital death after acute coronary syndromes (ACS), with its plateauing mortality rates approaching 50%. To test novel interventions, personalized risk prediction is essential. The ORBI (Observatoire Régional Breton sur l’Infarctus) score represents the first-of-its-kind risk score to predict in-hospital CS in ACS patients undergoing percutaneous coronary intervention (PCI). However, its sex-specific performance remains unknown, and refined risk prediction strategies are warranted.

Methods

This multinational study included a total of 53 537 ACS patients without CS on admission undergoing PCI. Following sex-specific evaluation of ORBI, regression and machine-learning models were used for variable selection and risk prediction. By combining best-performing models with highest-ranked predictors, SEX-SHOCK was developed, and internally and externally validated.

Results

The ORBI score showed lower discriminative performance for the prediction of CS in females than males in Swiss (area under the receiver operating characteristic curve [95% confidence interval]: 0.78 [0.76–0.81] vs. 0.81 [0.79–0.83]; P =.048) and French ACS patients (0.77 [0.74–0.81] vs. 0.84 [0.81–0.86]; P = .002). The newly developed SEX-SHOCK score, now incorporating ST-segment elevation, creatinine, C-reactive protein, and left ventricular ejection fraction, outperformed ORBI in both sexes (females: 0.81 [0.78–0.83]; males: 0.83 [0.82–0.85]; P < .001), which prevailed following internal and external validation in RICO (females: 0.82 [0.79–0.85]; males: 0.88 [0.86–0.89]; P < .001) and SPUM-ACS (females: 0.83 [0.77–0.90], P = .004; males: 0.83 [0.80–0.87], P = .001).

Conclusions

The ORBI score showed modest sex-specific performance. The novel SEX-SHOCK score provides superior performance in females and males across the entire spectrum of ACS, thus providing a basis for future interventional trials and contemporary ACS management.

Structured Graphical Abstract

This multinational study evaluates the sex-specific performance of the ORBI risk score in predicting in-hospital cardiogenic shock (CS) complicating acute coronary syndromes (ACS), and provides a novel score (i.e. SEX-SHOCK), now accounting for sex-specific disease and management characteristics. By leveraging machine learning (ML) and regression-based approaches, novel candidate predictors of CS were identified [i.e. creatinine, C-reactive protein (CRP), left ventricular ejection fraction (LVEF), and ST-segment elevation] and SEX-SHOCK was developed, and internally and externally validated. The SEX-SHOCK score outperforms ORBI in both sexes, showing improved performance for the prediction of in-hospital CS in females and males alike; thus, SEX-SHOCK mitigates sex inequities in the acute management of patients with ACS. AUC, area under the receiver operating characteristic curve; CAD, coronary artery disease; ECG, electrocardiogram; FHx, family history; FOR, false omission rate; HTN, hypertension; MLP, multiple layer perceptron; PCI, percutaneous coronary intervention; PPV, positive predictive value; RF, random forest; VD, vessel disease

Cardiogenic shock, Acute coronary syndromes, Atherosclerosis, Personalized risk prediction, Inflammation, C-reactive protein, LVEF, Percutaneous coronary intervention, Machine learning, Random forest, Multilayer perceptron, Gender medicine, Precision medicine

See the editorial comment for this article ‘The SEX-SHOCK score—the emperor's new clothes?’, by K.-P. Kresoja et al., https://doi.org/10.1093/eurheartj/ehae599.

Introduction

Acute coronary syndrome (ACS) continues to cause high morbidity and mortality across the globe. Of all patients presenting with ACS, 2%–10% develop cardiogenic shock (CS).¹ Despite the tremendous progress made in stabilized patients with ACS, mortality rates of CS plateaued at ∼50% 1 year after the index event.^2–4 The survival benefit conferred by mechanical circulatory support (MCS) remains controversial,^5,6 with international guidelines unequivocally supporting immediate revascularization of the infarct-related artery as the primary strategy to reduce CS-related mortality (class I recommendation).^7,8

The Society for Cardiovascular Angiography and Interventions (SCAI) proposed a three-axis model to risk stratify patients across the CS continuum, with increasing stages associating with higher mortality risk.⁹ While SCAI stage B is considered as the pre-shock phase, stage C is hallmarked by the presence of organ hypoperfusion with a dismal prognosis and very limited therapeutic options.^5,9 Assessing CS risk before hypoperfusion sets in may allow the implementation of therapeutic measures to prevent its progression to overt CS. This may represent a completely novel avenue to improve the management and outcomes of patients at high risk for the development of CS.

The Observatoire Régional Breton sur l’Infarctus (ORBI) score is the first risk score for the identification of ACS patients undergoing percutaneous coronary intervention (PCI) at risk of developing CS during hospital stay, thus enabling effective risk stratification according to individual susceptibilities for CS as a basis for contemporary management and future interventional trials.¹⁰ However, ORBI was developed in a predominantly male patient population and marked differences in ACS pathobiology between females and males may have insufficiently been accounted for.¹¹ Indeed, compared to their male counterparts, female ACS patients are older, have a higher comorbidity burden, experience longer pre-hospital delays, and are less likely to receive early revascularization or to be referred to tertiary-care shock centres, which is collectively linked to higher mortality risk.^12–15

In this large multinational study, we aimed (i) to assess the sex-specific performance of ORBI in predicting in-hospital CS in patients with ACS, and (ii) to develop a refined model on sex-disaggregated data to achieve refined risk prediction across the entire spectrum of ACS in females and males.

Methods

Study design and outcome definition

This is a retrospective analysis of existing cohort studies. In Switzerland, patient data were retrieved from two independent cohorts, namely the Acute Myocardial Infarction in Switzerland Plus (AMIS-Plus) study^16,17 and the Special Programme University Medicine Acute Coronary Syndrome (SPUM-ACS) study.^18–21 AMIS-Plus is a nationwide cohort study comprising 46 939 patients with ACS (recruitment period: 1 January 2005 until 27 August 2020), of which 35 650 underwent PCI. The SPUM-ACS study is an investigator-initiated prospective cohort study comprising a total of 4787 ACS patients presenting to any of the four major university hospitals in Switzerland (recruitment period: 8 December 2009 until 31 December 2017), of which 4186 underwent PCI. In France, patient data were retrieved from the obseRvatoire des Infarctus de Côte-d’Or (RICO) study which comprises 21 229 ACS patients recruited between 2001 and 2022,²² with 13 701 undergoing PCI. The study protocols of each cohort were approved by the local ethics committees and all study participants provided written informed consent. The primary endpoint was the occurrence of CS during initial hospitalization. Given the unavailability of (invasive) haemodynamic parameters and certain biomarkers in all-comers of patients with ACS, such as the cardiac index (<2.2 L/min/m²), pulmonary capillary wedge pressure (>15 mmHg), and lactate levels, in-hospital CS was defined as both a systolic blood pressure ≤ 90 mmHg after exclusion of hypovolaemia, and clinical signs of hypoperfusion, accompanied by the reliance on vasopressors/inotropic support or mechanical left ventricular assistance, as determined by treating physicians (see Supplementary data online, Table S1).^10,23 Patients already presenting with overt CS on admission were excluded from the analysis (see Supplementary data online, Figure S1).

Evaluation of model performance

Model discrimination was assessed separately for female and male patients using the area under the receiver operating characteristic (ROC) curve (AUC) and compared using the DeLong test for unpaired ROC curves. Model calibration was evaluated by the Brier score and calibration plots (see Supplementary data online, Figure S2). For the assessment of overall model performance, we computed the accuracy, false omission rate, sensitivity, specificity, positive predictive value, negative predictive value, and the F1 score.^24,25 To compare risk reclassification between SEX-SHOCK and ORBI, the integrated discrimination improvement and continuous net reclassification improvement were calculated. Decision curve analysis was conducted to compare the net benefit of the two models across different decision thresholds for predicting in-hospital CS.²⁶

Development and validation of SEX-SHOCK

Variable selection

A whole panel of variables, including clinical, biochemical, electrocardiographic, and imaging-derived variables, was selected based on clinical plausibility and data availability (see Supplementary data online, Table S2).^27,28 Predictive models were then built using logistic regression (LR) and machine-learning models, i.e. random forest (RF) and multilayer perceptron (MLP). Feature importance was assessed using tailored methods for each model. In LR models, Wald χ² minus degrees of freedom was used.¹⁹ In RF models, the Gini index served as the performance measure,²⁹ while for MLP, the permutation feature importance method was used, a proxy of the impact on model performance when features are shuffled.³⁰

Model selection and validation

Forward selection and backward elimination methods were employed in a sex-specific fashion to identify the optimal variable combination with the lowest Akaike Information Criteria.^25,31 The derivation cohort (AMIS-Plus) was randomly split into a training set (80% patients) and an internal testing set (20% patients). The training set was used to train RF, MLP, and LR models. Meanwhile, the internal testing set was utilized to assess their performance on unseen data and refine their hyperparameters. Following variable selection, LR and machine-learning-based models were compared to determine the best-performing modelling approach. Multicollinearity within the final model was assessed by the variance inflation factor and tolerance (see Supplementary data online, Table S3). Finally, SEX-SHOCK was internally validated using 10-fold cross-validation,³² with external validation being done in RICO (France) and SPUM-ACS (Switzerland).

Statistical analysis

Continuous variables are shown as median and interquartile range (IQR), while categorical data are presented as counts and valid percentages. Continuous variables were compared using Student’s t-test or Mann–Whitney test if non-normally distributed, and categorical data were analysed using χ², Fisher’s exact, or Kruskal–Wallis test, as appropriate. The degree of missing data is detailed in Supplementary data online, Table S4 (see Supplementary data online). To mitigate a potential missing data bias, multiple imputation using chained equations (MICE; n = 20 imputations) was performed for each cohort and sex separately. We employed predictive mean matching for continuous variables, proportional odds models for ordinal variables, and LR for binary variables, with in-hospital CS serving as the outcome variable.^33,34 Receiver operating characteristic curves and calibration plots were generated using a randomly selected dataset from the multiply imputed datasets. Finally, nomograms were constructed separately for each sex by converting the regression coefficients of multivariable-adjusted regression models proportionally to a 0–100-point scale. Total-point scores were obtained by summing the points assigned to each variable. Findings are reported in accordance with the guidelines set forth in the TRIPOD statement (see Supplementary data online, Figure S3) for transparent prediction model reporting and align with the standards of the STROBE initiative (see Supplementary data online, Figure S4). If not stated otherwise, a P < .05 was deemed significant. All analyses were performed in R (version 4.1.2) and IBM SPSS (version 27.0.1). Additional details on the variable and model selection process are provided in the Supplementary data online.

Results

Patients

A total of 35 650 ACS patients were included in AMIS-Plus, of which 8481 were female (24.80%). Female patients exhibited marked differences in baseline characteristics, ORBI components (Table 1), and ACS management as compared to males (see Supplementary data online, Table S5). They were older than males (median age: 71.5 [61.5, 79.3] vs. 62.5 [54.0, 72.1] years; P < .001) and the prevalence of previous stroke or transient ischaemic attack was higher (5.4% vs. 4.0%; P < .001). Additionally, females experienced longer pre-hospital delays relative to males (median: 420.0 [201.0, 1200.0] vs. 350.0 [172.0, 1012.0] min; P < .001). Females also tended to present with higher Killip classes (P < .001), although their systolic blood pressure levels were slightly higher (138.0 [120.0, 158.0] vs. 135.0 [120.0, 155.0]; P < .001). Blood glucose levels (median: 7.4 [6.2, 9.3] vs. 7.1 [6.1, 8.8] mmol/L; P < .001), and heart rates (median: 76.0 [66.0, 88.0] vs. 75.0 [65.0, 87.0] min⁻¹; P < .001) of female patients were also higher, suggesting an accentuated sympathetic response.

Table 1

Baseline characteristics of all patients stratified by sex in the nationwide AMIS-Plus study

	All patients (N = 35 650)	Females (N = 8481)	Males (N = 27 169)	P value
ORBI components
Age	64.5 [55.2, 74.4]	71.5 [61.5, 79.3]	62.5 [54.0, 72.1]	<.001
Presentation as cardiac arrest	1455 (4.1)	284 (3.3)	1171 (4.3)	<.001
Previous stroke/TIA	1529 (4.4)	451 (5.4)	1078 (4.0)	<.001
Anterior myocardial infarction	12 828 (36.5)	3077 (36.7)	9751 (36.4)	.566
First medical contact-to-PCI delay, min	365.0 [178.0, 1055.0]	420.0 [201.0, 1200.0]	350.0 [172.0, 1012.0]	<.001
Killip				<.001
I	32 190 (90.3)	7410 (87.4)	24 780 (91.2)
II	2503 (7.0)	759 (8.9)	1744 (6.4)
III	689 (1.9)	236 (2.8)	453 (1.7)
Heart rate, min−1	75.0 [65.0, 87.0]	76.0 [66.0, 88.0]	75.0 [65.0, 87.0]	<.001
Systolic blood pressure, mmHg	136.0 [120.0, 155.0]	138.0 [120.0, 158.0]	135.0 [120.0, 155.0]	<.001
Pulse pressure, mmHg	55.0 [43.0, 70.0]	60.0 [46.0, 75.0]	54.0 [42.0, 67.0]	<.001
Glucose, mmol/L	7.1 [6.1, 8.9]	7.4 [6.2, 9.3]	7.1 [6.1, 8.8]	<.001
TIMI flow post-PCI				.109
0	318 (1.3)	84 (1.4)	234 (1.2)
I	217 (0.9)	62 (1.1)	155 (0.8)
II	1057 (4.2)	258 (4.4)	799 (4.2)
III	23 345 (93.6)	5409 (93.1)	17 936 (93.8)
Candidate predictors
C-reactive protein, mg/L	4.0 [2.0, 9.0]	5.0 [2.0, 11.0]	4.0 [2.0, 9.0]	<.001
Creatinine, µmol/L	82.0 [70.0, 97.0]	72.0 [61.0, 87.0]	85.0 [74.0, 99.0]	<.001
ST-segment elevation	20 743 (58.2)	4877 (57.5)	15 866 (58.4)	.143
Left ventricular ejection fraction				.048
<35%	1777 (7.4)	462 (8.1)	1315 (7.2)
35%–50%	8607 (35.8)	2046 (36.0)	6561 (35.8)
>50%	13 626 (56.8)	3183 (55.9)	10 443 (57.0)
SCAI class
Aa	29 690 (85.6)	6824 (82.8)	22 866 (86.5)	<.001
Bb	5960 (16.7)	1657 (19.5)	4303 (15.8)	<.001
Biochemical and haemodynamic parameters
NT-proBNP, ng/L	898.0 [230.0, 2540.5]	1480.0 [459.0, 4333.5]	745.5 [191.0, 2117.0]	<.001
White blood cells, /μL	9800.0 [7800.0, 12 400.0]	9770.0 [7800.0, 12 300.0]	9810.0 [7800.0, 12 400.0]	.08
HbA1c, %	5.7 [5.4, 6.1]	5.7 [5.4, 6.1]	5.7 [5.4, 6.1]	.912
Haemoglobin, g/dL	14.4 [13.2, 15.4]	13.2 [12.2, 14.2]	14.7 [13.7, 15.7]	<.001
eGFR, mL/min/1.73 m2	81.6 [65.1, 94.5]	73.9 [56.7, 88.7]	83.8 [68.1, 95.9]	<.001
Diastolic blood pressure, mmHg	80.0 [70.0, 90.0]	78.0 [67.0, 88.0]	80.0 [70.0, 91.0]	<.001
Medical history
FHx of CAD (first degree relatives < 60 years)	10 146 (34.4)	2491 (36.2)	7655 (33.8)	<.001
Previous stable angina	5493 (15.7)	1244 (15.0)	4249 (15.9)	.038
Previous myocardial infarction	5300 (15.1)	983 (11.8)	4317 (16.2)	<.001
Previous PCI	5789 (16.5)	1072 (12.9)	4717 (17.7)	<.001
Previous CABG	1623 (4.6)	275 (3.3)	1348 (5.0)	<.001
Hypertension	20 770 (61.3)	5599 (69.0)	15 171 (58.8)	<.001
Diabetes	6476 (19.0)	1735 (21.5)	4741 (18.2)	<.001
Hypercholesterolaemia	20 337 (63.2)	4636 (61.1)	15 701 (63.8)	<.001
Comorbidities
Malignancy	1325 (3.8)	332 (4.0)	993 (3.7)	.274
Peripheral arterial diseases	1429 (4.1)	394 (4.7)	1035 (3.9)	.001
Hemiplegia	120 (0.3)	31 (0.4)	89 (0.3)	.67
Dementia	315 (0.9)	143 (1.7)	172 (0.6)	<.001
Chronic lung disease	1708 (4.9)	437 (5.3)	1271 (4.8)	.074
Connective tissue disease	426 (1.2)	204 (2.5)	222 (0.8)	<.001
Peptic ulcer disease	501 (1.4)	145 (1.7)	356 (1.3)	.007
Moderate to severe liver disease	166 (0.5)	35 (0.4)	131 (0.5)	.47
Moderate to severe renal disease	1840 (5.3)	568 (6.8)	1272 (4.8)	<.001
ECG on admission
Q-waves	2142 (6.0)	440 (5.2)	1702 (6.3)	<.001
ST-segment depression	8549 (24.0)	2210 (26.1)	6339 (23.3)	<.001
T-wave changes	6592 (18.5)	1759 (20.7)	4833 (17.8)	<.001
Left bundle branch block	890 (2.5)	249 (2.9)	641 (2.4)	.003
Right bundle branch block	1104 (3.1)	180 (2.1)	924 (3.4)	<.001
Type of vessel disease
1-VD	14 152 (40.1)	3576 (42.6)	10 576 (39.3)	<.001
2-VD	10 923 (30.9)	2548 (30.3)	8375 (31.1)	.183
3-VD	9929 (28.1)	2171 (25.8)	7758 (28.8)	<.001
LMCAD	592 (1.7)	135 (1.6)	457 (1.7)	.607
Culprit vessel				<.001
Left main	431 (2.1)	85 (1.8)	346 (2.3)
Left anterior descending artery (or one of its branches)	8448 (42.1)	2004 (41.8)	6444 (42.2)
Left circumflex artery (or one of its branches)	3737 (18.6)	828 (17.3)	2909 (19.0)
Right coronary artery (or one of its branches)	6937 (34.6)	1774 (37.0)	5163 (33.8)
Other	437 (2.2)	84 (1.8)	353 (2.3)
Type of MIc				<.001
Type 1	21 758 (92.3)	5204 (92.4)	16 554 (92.2)
Type 2	934 (4.0)	265 (4.7)	669 (3.7)
Type 3	11 (0.0)	5 (0.1)	6 (0.0)
Type 4a	81 (0.3)	17 (0.3)	64 (0.4)
Type 4b	724 (3.1)	129 (2.3)	595 (3.3)
Type 5	66 (0.3)	9 (0.2)	57 (0.3)
Location of MI
Inferior	13 152 (37.4)	3179 (38.0)	9973 (37.2)	.225
Posterior	3432 (9.8)	806 (9.7)	2626 (9.8)	.674
Lateral	3849 (15.7)	979 (16.7)	2870 (15.4)	.018
TIMI flow of culprit vessel pre-PCI				.239
0	7601 (54.6)	1763 (53.2)	5838 (55.0)
I	2861 (20.5)	719 (21.7)	2142 (20.2)
II	1546 (11.1)	360 (10.9)	1186 (11.2)
III	1925 (13.8)	473 (14.3)	1452 (13.7)
PCI complications
Myocardial infarction after PCI	140 (0.6)	34 (0.6)	106 (0.6)	.996
Emergency CABG after PCI	20 (0.1)	7 (0.1)	13 (0.1)	.368
Pericardiocentesis	42 (0.2)	18 (0.3)	24 (0.1)	.007
Intraprocedural death	64 (0.3)	28 (0.5)	36 (0.2)	<.001

Data are shown as median [IQR] or N (valid %).

CABG, coronary artery bypass grafting; eGFR, estimated glomerular filtration rate, calculated according to the Chronic Kidney Disease Epidemiology Collaboration (CKD-EPI) creatinine equation; FHx of CAD, family history of coronary artery disease; HbA1c, haemoglobin A1c; LMCAD, left main coronary artery disease; NT-proBNP, N-terminal pro b-type natriuretic peptide; PCI, percutaneous coronary intervention; TIA, transient ischaemic attack; TIMI, thrombolysis in myocardial infarction; 1/2/3 VD, 1/2/3 vessel disease.

^aDefined as warm and well-perfused with normal JVP (Killip I) and SBP ≥ 100 mmHg.

^bDefined as having elevated JVP (Killip II or higher), SBP < 90 mmHg, and/or no signs of classic CS (Killip IV).⁹

^cDefined according to the fourth universal definition of acute myocardial infarction.

Female patients were more likely to have impaired systolic function, as defined by left ventricular ejection fraction (LVEF) < 35%, as compared to their male counterparts (8.1% vs. 7.2%; P = .048). Women also had higher C-reactive protein (CRP) levels than males (median: 5.0 [2.0, 11.0] vs. 4.0 [2.0, 9.0] mg/L; P < .001), suggesting a greater inflammatory burden at the time of acute presentation. Despite lower creatinine levels among females, their estimated glomerular filtration rate (eGFR), a sex-adjusted measure of renal function, implied more severe renal impairment. In AMIS-Plus, 3.1% of all patients experienced in-hospital CS, with a higher relative incidence among females as compared to males (3.9% vs. 2.8%; P < .001). Sex-specific differences in baseline and management characteristics were similarly observed in RICO, in which a total of 13 701 patients were included. In these patients, 5.3% and 3.7% of female and male patients, respectively, developed in-hospital CS (P < .001) (see Supplementary data online, Tables S6 and S7).

Sex-specific performance of ORBI

While ORBI provided good discriminatory performance for the prediction of in-hospital CS in males (AUC [95% CI]: 0.81 [0.79–0.83]), its performance was lower in female patients recruited in Switzerland (0.78 [0.76–0.81]; P = .048) (Figure 1A). Similar results were obtained in French patients (males: 0.84 [0.81–0.86] vs. females: 0.77 [0.74–0.81]; P = .002) (Figure 1B). Indeed, in both Switzerland and France, ORBI performance among female ACS patients was characterized by higher Brier scores (i.e. a measure of prediction accuracy) and false omission rates (i.e. proportion of false negatives) as compared to males (see Supplementary data online, Table S8). Collectively, these findings indicate a limited sex-specific performance, with the ORBI risk score being more likely to miss true positives in females, thus systematically underestimating in-hospital CS risk in women.

Figure 1

Sex differences in ORBI performance in ACS patients undergoing PCI in Switzerland (left) and France (right). ROC curves of the ORBI risk score for the prediction of in-hospital cardiogenic shock are shown for female (red) and male patients (blue) in (A) AMIS-Plus (Switzerland) and (B) RICO (France). ROC curves were compared using an unpaired DeLong test. AMIS-Plus, Acute Myocardial Infarction in Switzerland Plus; AUC, area under the ROC curve; CI, confidence interval; ORBI, Observatoire Régional Breton sur l’Infarctus; PCI, percutanous coronary intervention; RICO, obseRvatoire des Infarctus de Côte-d’Or; ROC, receiver operating characteristic

Development of SEX-SHOCK

To address these limitations and consider sex-specific disease characteristics, we used machine-learning algorithms (i.e. RF and MLP) and LR on sex-disaggregated data and ranked potential predictors by feature importance separately for females and males (see Supplementary data online, Figure S5). The top 10 variables in females and males are depicted in Figure 2A–C. For females, top 10 variables across all modelling tactics tested included creatinine, CRP, LVEF, ST-segment elevation, and diabetes, while in males, CRP, LVEF, ST-segment elevation, and a history of dyslipidaemias provided marked predictive value towards incident CS. Finally, overlapping features (i.e. creatinine, CRP, LVEF, and ST-segment elevation) were selected as candidate variables to refine ORBI (Figure 2D, Supplementary data online, Figure S6). Forward selection and backward elimination were used to determine the optimal variable combination, with prior stroke/transient ischaemic attack, anterior ST-segment elevation myocardial infarction (STEMI), first medical contact-to-PCI delay, and Killip class II being replaced by creatinine, CRP, LVEF, and ST-segment elevation (see Supplementary data online, Tables S9 and S10). Among all model building approaches tested, LR emerged as the preferred method (see Supplementary data online, Figure S7), demonstrating highest predictive accuracy for both sexes. By combining best-performing variables with top-performing models, SEX-SHOCK was developed (see Supplementary data online, Figure S8).

Figure 2

Identification of most important predictors of in-hospital cardiogenic shock depending on sex. Top 10 variables identified by (A) logistic regression (LR), (B) random forest (RF), and (C) multilayer perceptron (MLP) in females (left; red) and males (right; blue). (D) Venn plots showing the intersection of highest-ranked predictors identified by LR, RF, and MLP in females (left; red) and males (right; blue). For females, the five overlapping variables include CRP, ST-segment elevation, LVEF, creatinine, and diabetes. For males, CRP, ST-segment elevation, history of dyslipidaemias, creatinine, and LVEF are among most important predictors across all model building approaches tested. LVEF was dummy coded in LR models. CAD, coronary artery disease; CRP, C-reactive protein; FHx, positive family history; HTN, history of hypertension; Hx, history of; LBBB, left bundle branch block; LMCAD, left main coronary artery disease; LVEF, left ventricular ejection fraction; MLP, multilayer perceptron; RBBB, right bundle branch block; RF, random forest; LR, logistic regression; 1-VD, single-vessel disease; 3-VD, three-vessel disease

Evaluation of SEX-SHOCK

Although relying on the identical number of predictors (n = 12), the discriminatory performance of SEX-SHOCK outperformed ORBI for the prediction of in-hospital CS in females (0.81 [0.78–0.83] vs. 0.78 [0.76–0.81], P < .001) and males alike (0.83 [0.82–0.85] vs. 0.81 [0.79–0.83], P < .001) (Figure 3A and B). SEX-SHOCK showed improved sensitivity, F1 score, false omission rate, and positive predictive value in both sexes (Figure 3C and D; Supplementary data online, Table S11). Decision curve analysis suggested that the net benefit of SEX-SHOCK at different decision thresholds surpassed that of ORBI in both sexes alike (Figure 4). Furthermore, irrespective of sex, SEX-SHOCK showed higher net reclassification and integrated discrimination improvement as compared to ORBI, emphasizing its superior performance as regards risk reclassification in both Swiss and French ACS patients (Table 2).

Figure 3

Performance of ORBI and SEX-SHOCK in the derivation cohort. ROC curves of the ORBI (black) and SEX-SHOCK score in (A) females (left; red) and (B) males (right; blue). ROC curves were compared using an unpaired DeLong test. Radar plots illustrate sensitivity, AUC, F1 score, false omission rate, and positive predictive value for the ORBI (grey area) and SEX-SHOCK score in (C) females (red area) and (D) males (blue area). AUC, area under the ROC curve; CI, confidence interval; FOR, false omission rate; ORBI, Observatoire Régional Breton sur l’Infarctus; PPV, positive predictive value; ROC receiver operating characteristic

Figure 4

Sex-stratified decision curve analysis comparing the SEX-SHOCK vs. ORBI risk score. Net benefit of the ORBI (black) and SEX-SHOCK score in predicting in-hospital cardiogenic shock in (A) females (left; red) and (B) males (right; blue) assuming that all (dashed grey line) or none (dashed back line) patients are at high risk across different risk thresholds. ORBI, Observatoire Régional Breton sur l’Infarctus

Table 2

Reclassification value of SEX-SHOCK vs. ORBI

Cohort	NRI (95% CI)	P value	IDI (95% CI)	P value
Females
AMIS-Plus	0.376 (0.267–0.484)	<.001	0.016 (0.009–0.024)	<.001
SPUM-ACS	0.485 (0.189–0.781)	.001	0.035 (0.003–0.068)	.031
RICO	0.500 (0.358–0.642)	<.001	0.033 (0.017–0.049)	<.001
Males
AMIS-Plus	0.323 (0.252–0.395)	<.001	0.016 (0.011–0.022)	<.001
SPUM-ACS	0.469 (0.313–0.625)	<.001	0.029 (0.013–0.044)	<.001
RICO	0.607 (0.507–0.706)	<.001	0.050 (0.037–0.063)	<.001

AMIS-Plus, Acute Myocardial Infarction in Switzerland Plus; CI, confidence interval; IDI, integrated discrimination improvement; NRI, net reclassification improvement; RICO, obseRvatoire des Infarctus de Côte-d’Or; SPUM-ACS, Special Programme University Medicine Acute Coronary Syndrome.

Internal and external validation of SEX-SHOCK

Following 10-fold cross-validation in AMIS-Plus (see Supplementary data online, Figure S9), the AUC for females ranged from 0.78 (95% CI, 0.67–0.89) to 0.91 (95% CI, 0.87–0.95), with a mean ± SD of 0.83 ± 0.05. In males, the AUC ranged from 0.82 (95% CI, 0.75–0.88) to 0.90 (95% CI, 0.85–0.94), with a mean ± SD of 0.86 ± 0.03. In both external validation cohorts (i.e. RICO and SPUM-ACS), SEX-SHOCK demonstrated superior discriminative performance as compared to ORBI (see Supplementary data online, Figure S10). Beyond the AUC, the sensitivity, the F1 score, and the positive predictive value were also improved, while false omission rate was reduced for both female and male patients (Figure 5, Supplementary data online, Table S10). Aligning with the data obtained in AMIS-Plus, decision curve analysis in both external validation cohorts suggested a greater net benefit of SEX-SHOCK in predicting in-hospital CS across various risk thresholds for both females and males (see Supplementary data online, Figure S11). To enhance the clinical applicability of the SEX-SHOCK score and allowing score calculation prior to PCI, a simplified model was developed, solely relying on non-procedural variables, showing similar performance to the full model (see Supplementary data online, Figure S12), while retaining its superiority as compared to ORBI in both validation cohorts (see Supplementary data online, Figure S13 and Supplementary data online, Table S12).

Figure 5

External validation of the newly developed SEX-SHOCK score. Radar plots showing the improved performance of the SEX-SHOCK score as compared to ORBI in terms of sensitivity, AUC, F1 score, false omission rate, and positive predictive value for females (red area) and males (blue area) in RICO (A, B) and SPUM-ACS (C, D). AUC, area under the receiver operating characteristic curve; FOR, false omission rate; PPV, positive predictive value; RICO, obseRvatoire des Infarctus de Côte-d’Or; SPUM-ACS, Special Programme University Medicine Acute Coronary Syndrome

Clinical application: nomogram of SEX-SHOCK

To allow for clinical use, sex-specific nomograms were developed for female and male ACS patients (Figure 6). Each predictor in SEX-SHOCK was assigned individual score points based on its individual contribution to overall CS risk. Individual score points were then summed to obtain a total score. Finally, using a function relating the total score to the probability of in-hospital CS, the predicted probability of in-hospital CS for each female or male ACS patient was calculated. Scores corresponding to different levels of each predictor used in the SEX-SHOCK model are detailed in Supplementary data online, Table S13 (see Supplementary data online). The online calculator for clinical use is available via www.mdcalc.com/calc/10563/sex-shock-risk-score-development-cardiogenic-shock.

Figure 6

Nomogram for refined risk prediction of cardiogenic shock in acute coronary syndromes: the SEX-SHOCK score. Nomogram to calculate the probability of in-hospital cardiogenic shock in (A) female and (B) male patients. Points: assigned scores for each predictor level. Total points: sum of individual score points across all predictors. Predicted probability of cardiogenic shock [Pr (CS)] is calculated based on the total score and the conversion relationship between the probability of the outcome event. Score points assigned to each predictor are summarized in Supplementary data online, Table S13 (see Supplementary data online). Given the skewed distribution of biomarker data, CRP and creatinine values were log-transformed. LVEF, left ventricular ejection fraction; PCI, percutaneous coronary intervention; PP, pulse pressure; Pr (CS), predicted probability of cardiogenic shock; SBP, systolic blood pressure; TIMI, thrombolysis in myocardial infarction grade

Discussion

Here, we demonstrate (i) that the ORBI risk score shows only modest performance in female ACS patients as compared to males, (ii) that CRP, LVEF, creatinine, and ST-segment elevation are potent predictors of in-hospital CS in both sexes, and (iii) that the newly developed SEX-SHOCK score, though relying on the identical number of variables, outperforms ORBI in both sexes across nations and clinical settings (Structured Graphical Abstract).

Currently available risk scores in the setting of CS, such as the IABP-SHOCK II,³⁵ ENCOURAGE,³⁶ SAVE,³⁷ and CARD-SHOCK score,³⁸ are primarily used to predict mortality and are applicable only to patients who present with, rather than being at risk of CS during hospitalization, thus serving solely as prognostic tools. Indeed, once ACS has progressed to overt CS (SCAI-C or higher), interventions tested so far might be implemented too late to change outcomes effectively. In fact, the efficacy and safety of mechanical or pharmacological support in reducing mortality in patients with established CS, despite one promising trial,³⁹ remains highly controversial, and novel risk stratification strategies are urgently warranted.^40–43 For instance, in both the DanGer SHOCK and ECLS-SHOCK trials, only patients with SCAI-C or higher were recruited, while patients with pre-hospital cardiac arrest were excluded from the former.^39,43 Hence, to reduce overall mortality, it might be worth considering applying therapeutic strategies early (e.g. in those at high CS risk but not yet in SCAI-C) with the goal of preventing CS and its progression into a refractory stage, in which patients have a dismal prognosis. In contrast to previous studies, the herein included derivation and validation cohorts also comprised patients with pre-hospital cardiac arrest and signs of myocardial ischaemia, with the SEX-SHOCK score being also applicable to these patients.

In daily clinical practice, patients in the pre-shock stage may be overlooked frequently due to the unavailability of quantifiable biomarkers for the differentiation between SCAI-A (at risk of CS) and SCAI-B (characterized by haemodynamic instability without organ hypoperfusion) and SCAI-C (organ hypoperfusion requiring pharmacologic or mechanical support).⁵ While soluble biomarkers of hypoperfusion, such as lactate, correlate well with short-term mortality in patients with overt CS,⁴⁴ normal lactate levels do not exclude the presence of haemodynamic instability.^45,46 By integrating clinical, biochemical, electrocardiographic, and imaging-derived features in a sex-specific fashion, SEX-SHOCK is the first internally and externally validated risk score to precisely estimate CS risk in the pre-shock phase in both females and males, potentially allowing timely identification of high-risk patients who may benefit from novel interventions to prevent the progression to overt CS.

For instance, LVEF, an important imaging parameter linked to adverse events in patients with CS, represents an important parameter to determine a patient’s benefit from MCS and guiding treatment strategies to optimize expected benefits.⁴⁷ Additionally, worsening renal function serves as an important proxy for end-organ hypoperfusion and has been incorporated into various CS scoring systems previously.^35,36,48,49 Similarly, systemic inflammation plays a crucial role in CS pathobiology, contributing to its progression,^50–52 with CS patients displaying higher levels of inflammatory markers [e.g. CRP, tumour necrosis factor α, and interleukin (IL)-6] as compared to controls, which may be linked to poor outcomes.^51,53–55 Notably, anti-inflammatory therapy by IL-1β inhibition reduces total cardiovascular events in stabilized patients with prior ACS and high residual inflammatory burden,^56,57 although the benefits of anti-inflammatory therapies for the prevention of CS development in ACS patients remains to be comprehensively investigated.

Of note, CS patients with non-ST-segment elevation ACS (NSTE-ACS) have a higher baseline risk profile than those with STEMI, with CS complicating NSTE-ACS typically occurring after a median of 76–94 h.^58,59 Despite this, NSTE-ACS patients, whether they have established CS or not, undergo coronary angiography less frequently compared to STEMI patients, particularly if they are female.^58,60 Moreover, although women present with NSTE-ACS more often, they receive timely guideline-recommended care less frequently as compared to males.⁶¹

Hence, objective risk assessment is particularly important for the management of female ACS patients, as these patients are older, have higher comorbidity burden, experience longer pre-hospital delays, are less likely to be referred to tertiary-care shock centres, and to receive early revascularization,^15,62 making an optimal approach to a personalized treatment strategy challenging. The novel SEX-SHOCK score was trained and validated on sex-disaggregated data, provides objective risk assessment, and thus may mitigate sex inequities in the acute management of patients across the entire spectrum of ACS.

Strengths and limitations

Our study has several strengths. First, we analysed one of the largest and best characterized patient cohorts on ACS and CS in Europe, with a total sample size exceeding most previous studies on risk prediction in CS. Second, patients enrolled between 2005 and 2022 were analysed, accounting for the evolution of both ACS and CS phenotypes and thus reflecting evolving strategies of contemporary ACS management. Third, we used two different external validation cohorts, allowing to test the performance of SEX-SHOCK across healthcare systems, nations, and clinical settings.

Despite these strengths, certain limitations warrant discussion. First, the sex-specific differences in ORBI score performance were only modest in magnitude in AMIS-Plus. Second, although the superior performance of SEX-SHOCK in both validation cohorts argues for a high predictive utility of CRP, data on this biomarker were only available in 67.6% of patients in derivation cohort. Additionally, data on initial lactate levels were unavailable in the derivation and validation cohorts; thus, future studies should assess whether the integration of biomarker data beyond CRP and creatinine can further improve SEX-SHOCK score performance.⁴⁶ Along similar lines, given the unavailability of patients’ ethnicity in the derivation cohort, additional studies might be warranted to assess the generalizability of the herein reported results across social–cultural aspects. Third, we did not assess the predictive performance of SEX-SHOCK over time (from study inclusion to discharge) due to unavailability of data on the exact time point of in-hospital CS. Indeed, the latter represents a major limitation of the present study, as certain patients may have moved to a higher SCAI class before all variables informing SEX-SHOCK were available. Fourth, as certain patients (e.g. those with pre-hospital cardiac arrest or those presenting in SCAI-B) might be underrepresented in the present study, independent validation studies are certainly warranted to probe score performance across patient subgroups and CS entities. Fifth, whether the clinically relevant improvements in risk prediction of SEX-SHOCK reflect into improved outcomes of ACS patients at risk of developing CS needs to be demonstrated in well-designed interventional trials. Finally, our study has certain limitations inherent to its observational design, including residual confounding. However, we would argue that our study results could inform the design of future interventional trials, focusing on a patient population at risk of rather than fully established CS.

Conclusions

By integrating best-performing models with highest-ranked predictors, the SEX-SHOCK score demonstrates excellent discriminatory performance for the prediction of in-hospital CS in both females and males across the entire spectrum of ACS, thus mitigating sex inequities in early risk stratification of contemporary ACS management. The SEX-SHOCK score facilitates the early identification of ACS patients at high risk of CS and may guide contemporary clinical decision-making and patient selection for future randomized controlled trials.

Acknowledgements

We wish to thank the steering committee members of all study cohorts, the local study nurses, the core lab technicians, the local catheter teams, and the central data monitors for supervising the electronic data capturing system of each cohort. Specifically, we wish to thank the Clinical Trial Unit at the University of Bern and the AMIS-Plus Data Center at the University of Zurich, and all physicians and clinical research assistants that ensured data collection and monitoring of the RICO study.

Supplementary data

Supplementary data are available at European Heart Journal online.

Declarations

Disclosure of Interest

M.Z. declares research grants from Amarin Corp and lecture fees from Organon, Amgen and Pfizer. V.A. declares lecture fees from Bouchara-Recordati, BMS-Pfizer, Edwards Lifescience and Medtronic, and consulting fees from Boston Scientific and Medtronic. All other authors report no conflict of interest. M.R. reports institutional research grants from Terumo, Biotronik, and Cordis. L.L. and G.G.C. are coinventors on the International Patent WO/2020/226993 filed in April 2020 unrelated to this work. L.L. reports speaker fees outside of this work from Daiichi-Sankyo. F.C. reports having received non-financial support and speaking fees for Amgen, MSD, Novartis, Sanofi, and Pfizer. Y.C. reports having received consultant or speaking fees for Bayer, BMS/Pfizer, Boehringer Ingelheim, Novartis, Sanofi and Servier. T.F.L. has no conflicts of interest related to this work but has received educational and research grants to the institution from Abbott, Amgen, AstraZeneca, Boehringer Ingelheim, Daiichi Sankyo, Eli Lilly, Novartis, Novo Nordisk, Sanofi and Vifor and consulting fees from Abbott India, Daiichi Sankyo, Milestone Pharmaceuticals and Novo Nordisk. T.F.L. holds leadership positions at the European Society of Cardiology, Swiss Heart Foundation, and the Foundation for Cardiovascular Research—Zurich Heart House, Zurich, Switzerland and the London Heart house in London, UK. S.K. declares outside this work speaker fees from Roche Diagnostics and the Foundation for Cardiovascular Research – Zurich Heart House. Further, he has received research grants to the institution from the Jubiläumsstiftung SwissLife, the Lindenhof Foundation, the Novartis Foundation for Medical-biological Research, the Swiss Heart Foundation, the Swiss Society of Cardiology, and the Theodor-Ida-Herzog-Egli Foundation, and equipment and materials from Roche Diagnostics outside the submitted work. Travel support, again unrelated to this work, was received from the European Atherosclerosis Society, the European Society of Cardiology, the European Society of Clinical Investigation, Sphingotec GmbH, the 4TEEN4 Pharmaceuticals GmbH, and PAM Theragnostics GmbH. The other authors declare no disclosure of interest related to this manuscript.

Data Availability

Due to strict data protection regulations, the authors do not have authorisation to provide unrestricted data access. Data requests from qualified investigators can be made to the corresponding authors and will be considered by the SPUM-ACS', AMIS-Plus' and RICO' steering committees, subject to institutional and ethical committee approvals.

Funding

This work was supported by funding granted by the Swiss National Research Foundation (SPUM 33CM30-124112 and 32473B_163271; to T.F.L.), the Novartis Foundation for Medical-Biological Research (to S.K.), the Swiss Heart Foundation (to S.K., T.F.L.), the Research Prize of the Swiss Society of Cardiology (to S.K.), the Jubiläumsstiftung SwissLife (to S.K.), the Foundation for Cardiovascular Research – Zurich Heart House, and the China Scholarship Council grant (to Y.W.). The RICO study was supported by the University Hospital of Dijon Bourgogne, the Association de Cardiologie de Bourgogne, Fédération Française de Cardiologie, and by grants from the Agence Régionale de Santé de Bourgogne Franche-Comté, from the Conseil Régional de Bourgogne Franche-Comté.

Ethical Approval

The study protocols of each cohort were approved by the local ethics committees and all study participants provided written informed consent.

Pre-registered Clinical Trial Number

AMIS-Plus (NCT01305785), SPUM-ACS (NCT01000701).

Author notes

Yifan Wang first author.

Thomas F. Lüscher and Simon Kraler jointly directed the study.