-
PDF
- Split View
-
Views
-
Cite
Cite
Rakesh Malhotra, Kianoush B. Kashani, Etienne Macedo, Jihoon Kim, Josee Bouchard, Susan Wynn, Guangxi Li, Lucila Ohno-Machado, Ravindra Mehta, A risk prediction score for acute kidney injury in the intensive care unit, Nephrology Dialysis Transplantation, Volume 32, Issue 5, May 2017, Pages 814–822, https://doi.org/10.1093/ndt/gfx026
- Share Icon Share
ABSTRACT
Background. Acute kidney injury (AKI) is common in critically ill patients and is associated with high morbidity and mortality. Early identification of high-risk patients provides an opportunity to develop strategies for prevention, early diagnosis and treatment of AKI.
Methods. We undertook this multicenter prospective cohort study to develop and validate a risk score for predicting AKI in patients admitted to an intensive care unit (ICU). Patients were screened for predictor variables within 48 h of ICU admission. Baseline and acute risk factors were recorded at the time of screening and serum creatinine was measured daily for up to 7 days. A risk score model for AKI was developed with multivariate regression analysis combining baseline and acute risk factors in the development cohort (573 patients) and the model was further evaluated on a test cohort (144 patients). Validation was performed on an independent prospective cohort of 1300 patients. The discriminative ability of the risk model was assessed by the area under the receiver operating characteristic curve (AUROC) and model calibration was evaluated by Hosmer–Lemeshow statistic. AKI was defined by the Kidney Disease: Improving Global Outcomes criteria (absolute change of 0.3 mg/dL or relative change of 50% from baseline serum creatinine in 48 h to 7 days, respectively).
Results. AKI developed in 754 (37.2%) patients. In the multivariate model, chronic kidney disease, chronic liver disease, congestive heart failure, hypertension, atherosclerotic coronary vascular disease, pH ≤ 7.30, nephrotoxin exposure, sepsis, mechanical ventilation and anemia were identified as independent predictors of AKI and the AUROC for the model in the test cohort was 0.79 [95% confidence interval (CI) 0.70–0.89]. On the external validation cohort, the AUROC value was 0.81 (95% CI 0.78–0.83). The risk model demonstrated good calibration in both cohorts. Positive and negative predictive values for the optimal cutoff value of ≥ 5 points in test and validation cohorts were 22.7 and 96.1% and 31.8 and 95.4%, respectively.
Conclusions. A risk score model integrating chronic comorbidities and acute events at ICU admission can identify patients at high risk to develop AKI. This risk assessment tool could help clinicians to stratify patients for primary prevention, surveillance and early therapeutic intervention to improve care and outcomes of ICU patients.
INTRODUCTION
Acute kidney injury (AKI) is a life-threatening and disabling complication of critical illnesses encountered in 25–50% of intensive care unit (ICU) admissions [1–3]. Several studies have established the relationship between small increments in serum creatinine (SCr) and adverse events and suggested that accurate identification of individuals at risk and early recognition of AKI episodes could offer opportunities for diagnostic, preventive or even therapeutic interventions [4–8]. Over the past decade, several risk stratification scores have been developed to predict AKI in specific clinical settings (e.g. after cardiac surgery, contrast exposure, hospital acquired, general surgery and high-risk surgery) [9–22]. There are also few models examining the clinical risk factors for the development of AKI in the ICU population [23–30]. However, these risk assessment tools were limited to single centers, small sample sizes and with no internal or external validation.
Recent studies have focused on the use of biomarkers of kidney injury to identify patients at increased risk but mostly have not integrated these with clinical risk models [31–38]. Basu et al. [39] recently showed that combining clinical data with plasma biomarkers can improve the accuracy of risk prediction of severe AKI in pediatric ICU patients. We believe that there is a need to develop improved clinical risk prediction tools for AKI in the adult ICU setting to provide clinicians actionable information for prevention, early diagnosis and targeted interventions. We hypothesized that a risk stratification score based on routinely available clinical variables would accurately predict the risk for AKI in an ICU population.
MATERIALS AND METHODS
This is a prospective observational study conducted at two large, tertiary care university hospitals. The Institutional Review Board committees at the University of California, San Diego (UCSD), San Diego, CA and Mayo Clinic, Rochester, MN, USA approved the study. Informed consent was obtained from each participant in the UCSD cohort and was waived for patients who provided research authorization in the Mayo cohort. We excluded patients without research authorization in the Mayo cohort.
Study population

(A) Screening and enrollement procedures and (B) Flow chart of Study.
Data collection and outcomes
We recorded therapeutic regimens and demographic, anthropometric, clinical and laboratory data from the electronic health records at the time of screening, i.e. within 48 h of ICU admission (Figure 1A). Each institution’s local laboratory measured SCr values using the Jaffe alkaline picrate method. The primary outcome variable measured was the development of AKI, defined as an absolute increase in SCr level of ≥0.3 mg/dL within 48 h or ≥50% above the reference value within 7 days after study enrollment [40]. We considered the mean of all SCr measurements 7–365 days prior to admission as the reference SCr. We used SCr measured at ICU admission as an imputation value in patients with missing baseline SCr [24% (n = 172) in the UCSD cohort and 30% (n = 384) in the Mayo cohort]. Secondary outcome variables were lengths of stay (LOSs) in the ICU and hospital, need for renal replacement therapy (RRT) and ICU mortality.
Risk factor profiling
Risk factors were classified as chronic comorbidities [advanced age (>70 years), diabetes mellitus (DM), atherosclerotic coronary vascular disease (ASCVD), congestive heart failure (CHF), CKD, chronic liver disease, hypertension (HTN), obesity, cancer, drug abuse, cerebrovascular accident, human immunodeficiency virus or chronic lung disease] and acute events [i.e. hypotension (mean arterial pressure < 70 mmHg or use of any vasopressor)], sepsis, high-risk surgery, (cardiac surgery, including valvular or coronary artery bypass grafting, aortic surgery and hepatobiliary surgery), mechanical ventilation (MV), traumatic brain injury, rhabdomyolysis, anemia (hemoglobin < 9 mg/dL or hematocrit < 27%), hyperglycemia (blood glucose >120 mg/dL, excluding DM), elevated bilirubin (total bilirubin > 2 mg/dL, excluding chronic liver disease), decreased albumin (serum albumin < 3 mg/dL), low blood pH ( ≤ 7.30) or nephrotoxin exposure (Supplementary data, Table E1). The chronic and acute risk factors were evaluated at the time of screening, i.e. within 48 h of ICU admission. The chronic comorbidities were collected from the electronic health records problem list. The risk factors were chosen based on previous studies (Supplementary data, Table E1) [30, 41–59]. Continuous predictor variables were converted into categorical ones based on optimal clinical cutoff points used in the AKI literature. We have chosen categorization, as it is easier to interpret and also the simplicity of reporting results. In total, 25 binary predictor variables and 4 continuous predictor variables [age, first SCr at ICU admission, body surface area (BSA) and body mass index (BMI)] were selected for model development. Risk factor data were missing in <5% of patients and were imputed in none.
Statistical analysis
Continuous variables were expressed as the mean (SD) or median and interquartile range and analyzed by unpaired t-test or the Wilcoxon rank-sum test, as appropriate. Categorical variables are expressed as absolute (n) and relative (%) frequency and analyzed by chi-square test or Fisher’s exact test, as appropriate.
Risk model development and validation
Using the development cohort at UCSD, the regression coefficient-based models were constructed using a 5-fold cross-validation procedure [60, 61]. Of the 717 patients from the UCSD cohort included in the analysis, we randomly split the data into five mutually exclusive partitions of nearly equal size (n ∼ 144) with a stratified sampling to keep the ratio between AKI outcome positives and negatives identical across five partitions. In a preprocessing step, categorical variables were prescreened using chi-square test (P < 0.1). Stepwise forward elimination was used for variable selection in the multiple logistic regression. Selected predictor variables were checked for collinearity and interactions. For round 1, we held the partition 1 as a test set, trained a model on the other four partitions and calculated two performance measures, the area under the receiver operating characteristic (AUROC) value and the Hosmer–Lemeshow (HL) P-value, on partition 1, unused in the model training. For round 2, we held the partition 2 as a test set, trained a model on the other four partitions and calculated two performance metrics, the AUROC and HL P-value, on partition 2, unused in the model training. We repeated this five times to have five candidate models with a corresponding five pairs of AUROC values and HL P-values. Of these, the second candidate model of the five (Model 2), which had the largest AUROC and passing HL test, was selected as the final model. The coefficients generated for each variable in the final multivariate model was rounded to the nearest integer for the development of an easy-to-use AKI risk score. By summing the component variables together, the total score can range from a minimum of 0 to a maximum of 21 points. Subsequently, the final AKI risk score model was assessed in the Mayo Clinic validation cohort using the AUROC curve C-statistic and HL goodness-of-fit test.
The optimal cutoff point for the continuous risk score was determined using the highest Youden’s index, which is defined as J = maximum [sensitivity + specificity − 1], calculated from the AUROC analysis [62]. The definitions of risk factors and clinical outcomes used were similar in the UCSD and Mayo cohorts (Supplementary data, Table E1). Statistical analysis was performed using SPSS software version 17.0 (SPSS, Chicago, IL, USA) and R version 3.2.1 (R Project for Statistical Computing, Vienna, Austria).
RESULTS
Study cohort characteristics and outcomes
Clinical and demographic characteristics of both cohorts are summarized in Table 1. In the UCSD cohort, 717 patients enrolled with a mean age of 54 (SD 18) years; 63% (n = 453) were men and 58% (n = 413) were white. The Mayo Clinic cohort consisted of 1303 patients with a mean age of 63 (SD 20) years; 53% (n = 687) were men and 97% (n = 1268) were white. The prevalence of HTN in the UCSD and Mayo Clinic cohorts was 35% (n = 254) and 61% (n = 796), respectively. In total, 42% (n = 301) of UCSD patients and 27% (357) of Mayo Clinic patients were on MV. The overall incidence of AKI was 37.2% (n = 754); stage 1, 39.5% (n = 298); stage 2, 35.5% (n = 268); stage 3, 7.7% (n = 58) and 15.5% (n = 117) requiring dialysis. The median time to develop AKI from study enrollment was 24.3 h [95% confidence interval (CI) 12.2–54.7; 23.2 h (12.4–48.4) in the UCSD and 24.4 h (12.0–60.1) in Mayo Clinic cohort, respectively]. The overall ICU mortality rate was 6.3% [7% (n = 52) and 6% (n = 76) in the UCSD and Mayo Clinic cohorts, respectively]. The patients who developed AKI had significantly higher ICU mortality rates than those without AKI [n = 77 (10%) versus n = 51 (4%); P ≤ 0.001].
Demographic and outcome characteristics of the UCSD development, UCSD test and Mayo Clinic validation cohorts
Variables . | UCSD training set (A) (n = 573) . | UCSD test set (B) (n = 144) . | Mayo Clinic validation set (C) (n = 1300) . |
---|---|---|---|
CCU and MICU, n (%) | 277 (48) | 71 (49) | 786 (61) |
SICU, n (%) | 296 (52) | 73 (51) | 514 (40) |
Age, years, mean (SD) | 54 (18) | 54.2 (18) | 65 (48–78) |
Race, white, n (%) | 337 (59) | 76 (53) | 1265 (97) |
Male, n (%) | 367 (64) | 86 (60) | 686 (53) |
BSA, m2, median (IQR) | 1.9 (1.7–2.1) | 1.9 (1.7–2.1) | 1.8 (1.6–1.9) |
BMI, kg/m2, median (IQR) | 26 (23–31) | 25 (23–29) | 25 (22–30) |
SCr at ICU admission, mg/dL, median (IQR) | 0.9 (0.7–1.1) | 0.9 (0.7–1.0) | 1.0 (0.8–1.2) |
Age >70, years, mean (SD) | 120 (21) | 23 (16) | 670 (51) |
Diabetes, n (%) | 143 (25) | 44 (31) | 352 (27) |
Hypertension, n (%) | 207 (36) | 47 (33) | 793 (61) |
Morbid obesity, n (%) | 155 (27) | 28 (19) | 311 (24) |
Chronic liver disease, n (%) | 61 (11) | 23 (16) | 70 (5) |
Congestive heart failure, n (%) | 77 (13) | 20 (14) | 166 (13) |
Chronic lung disease, n (%) | 114 (20) | 32 (22) | 81 (6) |
Chronic kidney disease, n (%) | 43 (8) | 8 (6) | 144 (11) |
Hypotension, n (%) | 154 (27) | 46 (32) | 103 (8) |
Mechanical ventilation, n (%) | 238 (42) | 63 (44) | 357 (27) |
pH value ≤7.30, n (%) | 79 (14) | 13 (9) | 172 (13) |
Severe infection/sepsis, n (%) | 93 (16) | 26 (18) | 463 (36) |
Nephrotoxin exposure, n (%) | 114 (20) | 30 (21) | 96 (7) |
Incidence of AKI, n (%) | 127 (22) | 35 (24) | 590 (45) |
Need for RRT, n (%) | 30 (5) | 10 (7) | 77 (5) |
ICU mortality, n (%) | 41 (7) | 11 (8) | 76 (6) |
ICU stay, days, median (IQR) | 3 (2–5) | 3 (2–7) | 1 (1–2) |
Hospital stay, days, median (IQR) | 6 (3–13) | 10 (4–24) | 5 (3–8) |
Variables . | UCSD training set (A) (n = 573) . | UCSD test set (B) (n = 144) . | Mayo Clinic validation set (C) (n = 1300) . |
---|---|---|---|
CCU and MICU, n (%) | 277 (48) | 71 (49) | 786 (61) |
SICU, n (%) | 296 (52) | 73 (51) | 514 (40) |
Age, years, mean (SD) | 54 (18) | 54.2 (18) | 65 (48–78) |
Race, white, n (%) | 337 (59) | 76 (53) | 1265 (97) |
Male, n (%) | 367 (64) | 86 (60) | 686 (53) |
BSA, m2, median (IQR) | 1.9 (1.7–2.1) | 1.9 (1.7–2.1) | 1.8 (1.6–1.9) |
BMI, kg/m2, median (IQR) | 26 (23–31) | 25 (23–29) | 25 (22–30) |
SCr at ICU admission, mg/dL, median (IQR) | 0.9 (0.7–1.1) | 0.9 (0.7–1.0) | 1.0 (0.8–1.2) |
Age >70, years, mean (SD) | 120 (21) | 23 (16) | 670 (51) |
Diabetes, n (%) | 143 (25) | 44 (31) | 352 (27) |
Hypertension, n (%) | 207 (36) | 47 (33) | 793 (61) |
Morbid obesity, n (%) | 155 (27) | 28 (19) | 311 (24) |
Chronic liver disease, n (%) | 61 (11) | 23 (16) | 70 (5) |
Congestive heart failure, n (%) | 77 (13) | 20 (14) | 166 (13) |
Chronic lung disease, n (%) | 114 (20) | 32 (22) | 81 (6) |
Chronic kidney disease, n (%) | 43 (8) | 8 (6) | 144 (11) |
Hypotension, n (%) | 154 (27) | 46 (32) | 103 (8) |
Mechanical ventilation, n (%) | 238 (42) | 63 (44) | 357 (27) |
pH value ≤7.30, n (%) | 79 (14) | 13 (9) | 172 (13) |
Severe infection/sepsis, n (%) | 93 (16) | 26 (18) | 463 (36) |
Nephrotoxin exposure, n (%) | 114 (20) | 30 (21) | 96 (7) |
Incidence of AKI, n (%) | 127 (22) | 35 (24) | 590 (45) |
Need for RRT, n (%) | 30 (5) | 10 (7) | 77 (5) |
ICU mortality, n (%) | 41 (7) | 11 (8) | 76 (6) |
ICU stay, days, median (IQR) | 3 (2–5) | 3 (2–7) | 1 (1–2) |
Hospital stay, days, median (IQR) | 6 (3–13) | 10 (4–24) | 5 (3–8) |
CCU, critical care unit.
Demographic and outcome characteristics of the UCSD development, UCSD test and Mayo Clinic validation cohorts
Variables . | UCSD training set (A) (n = 573) . | UCSD test set (B) (n = 144) . | Mayo Clinic validation set (C) (n = 1300) . |
---|---|---|---|
CCU and MICU, n (%) | 277 (48) | 71 (49) | 786 (61) |
SICU, n (%) | 296 (52) | 73 (51) | 514 (40) |
Age, years, mean (SD) | 54 (18) | 54.2 (18) | 65 (48–78) |
Race, white, n (%) | 337 (59) | 76 (53) | 1265 (97) |
Male, n (%) | 367 (64) | 86 (60) | 686 (53) |
BSA, m2, median (IQR) | 1.9 (1.7–2.1) | 1.9 (1.7–2.1) | 1.8 (1.6–1.9) |
BMI, kg/m2, median (IQR) | 26 (23–31) | 25 (23–29) | 25 (22–30) |
SCr at ICU admission, mg/dL, median (IQR) | 0.9 (0.7–1.1) | 0.9 (0.7–1.0) | 1.0 (0.8–1.2) |
Age >70, years, mean (SD) | 120 (21) | 23 (16) | 670 (51) |
Diabetes, n (%) | 143 (25) | 44 (31) | 352 (27) |
Hypertension, n (%) | 207 (36) | 47 (33) | 793 (61) |
Morbid obesity, n (%) | 155 (27) | 28 (19) | 311 (24) |
Chronic liver disease, n (%) | 61 (11) | 23 (16) | 70 (5) |
Congestive heart failure, n (%) | 77 (13) | 20 (14) | 166 (13) |
Chronic lung disease, n (%) | 114 (20) | 32 (22) | 81 (6) |
Chronic kidney disease, n (%) | 43 (8) | 8 (6) | 144 (11) |
Hypotension, n (%) | 154 (27) | 46 (32) | 103 (8) |
Mechanical ventilation, n (%) | 238 (42) | 63 (44) | 357 (27) |
pH value ≤7.30, n (%) | 79 (14) | 13 (9) | 172 (13) |
Severe infection/sepsis, n (%) | 93 (16) | 26 (18) | 463 (36) |
Nephrotoxin exposure, n (%) | 114 (20) | 30 (21) | 96 (7) |
Incidence of AKI, n (%) | 127 (22) | 35 (24) | 590 (45) |
Need for RRT, n (%) | 30 (5) | 10 (7) | 77 (5) |
ICU mortality, n (%) | 41 (7) | 11 (8) | 76 (6) |
ICU stay, days, median (IQR) | 3 (2–5) | 3 (2–7) | 1 (1–2) |
Hospital stay, days, median (IQR) | 6 (3–13) | 10 (4–24) | 5 (3–8) |
Variables . | UCSD training set (A) (n = 573) . | UCSD test set (B) (n = 144) . | Mayo Clinic validation set (C) (n = 1300) . |
---|---|---|---|
CCU and MICU, n (%) | 277 (48) | 71 (49) | 786 (61) |
SICU, n (%) | 296 (52) | 73 (51) | 514 (40) |
Age, years, mean (SD) | 54 (18) | 54.2 (18) | 65 (48–78) |
Race, white, n (%) | 337 (59) | 76 (53) | 1265 (97) |
Male, n (%) | 367 (64) | 86 (60) | 686 (53) |
BSA, m2, median (IQR) | 1.9 (1.7–2.1) | 1.9 (1.7–2.1) | 1.8 (1.6–1.9) |
BMI, kg/m2, median (IQR) | 26 (23–31) | 25 (23–29) | 25 (22–30) |
SCr at ICU admission, mg/dL, median (IQR) | 0.9 (0.7–1.1) | 0.9 (0.7–1.0) | 1.0 (0.8–1.2) |
Age >70, years, mean (SD) | 120 (21) | 23 (16) | 670 (51) |
Diabetes, n (%) | 143 (25) | 44 (31) | 352 (27) |
Hypertension, n (%) | 207 (36) | 47 (33) | 793 (61) |
Morbid obesity, n (%) | 155 (27) | 28 (19) | 311 (24) |
Chronic liver disease, n (%) | 61 (11) | 23 (16) | 70 (5) |
Congestive heart failure, n (%) | 77 (13) | 20 (14) | 166 (13) |
Chronic lung disease, n (%) | 114 (20) | 32 (22) | 81 (6) |
Chronic kidney disease, n (%) | 43 (8) | 8 (6) | 144 (11) |
Hypotension, n (%) | 154 (27) | 46 (32) | 103 (8) |
Mechanical ventilation, n (%) | 238 (42) | 63 (44) | 357 (27) |
pH value ≤7.30, n (%) | 79 (14) | 13 (9) | 172 (13) |
Severe infection/sepsis, n (%) | 93 (16) | 26 (18) | 463 (36) |
Nephrotoxin exposure, n (%) | 114 (20) | 30 (21) | 96 (7) |
Incidence of AKI, n (%) | 127 (22) | 35 (24) | 590 (45) |
Need for RRT, n (%) | 30 (5) | 10 (7) | 77 (5) |
ICU mortality, n (%) | 41 (7) | 11 (8) | 76 (6) |
ICU stay, days, median (IQR) | 3 (2–5) | 3 (2–7) | 1 (1–2) |
Hospital stay, days, median (IQR) | 6 (3–13) | 10 (4–24) | 5 (3–8) |
CCU, critical care unit.
Development of a risk prediction model for AKI (UCSD cohort)

Discriminative ability of the five candidate models for the risk prediction of acute kidney injury expressed as AUC for the UCSD test cohort (n = 144 patients). The one with the largest AUC, Model 2, is drawn in solid and bold line.
Predictors of AKI obtained by binary multivariate logistic regression analysis in the UCSD development cohort (n = 573 patients)
Comorbidities . | Coefficient . | OR . | 95% CI for OR . | P-value . | |
---|---|---|---|---|---|
. | . | . | Lower . | Upper . | . |
pH ≤ 7.30 | 0.977 | 2.656 | 1.436 | 4.916 | 0.002 |
Nephrotoxin exposure | 0.929 | 2.532 | 1.497 | 4.281 | 0.001 |
Chronic kidney disease | 0.860 | 2.363 | 1.153 | 4.842 | 0.02 |
Chronic liver disease | 0.778 | 2.177 | 1.118 | 4.239 | 0.02 |
Severe infection/sepsis | 0.743 | 2.102 | 1.175 | 3.763 | 0.01 |
Congestive heart failure | 0.720 | 2.054 | 1.059 | 3.985 | 0.03 |
Hypertension | 0.563 | 1.756 | 1.084 | 2.844 | 0.02 |
Atherosclerotic coronary vascular disease | 0.490 | 1.632 | 0.905 | 2.945 | 0.10 |
Mechanical ventilation | 0.447 | 1.564 | 0.981 | 2.493 | 0.06 |
Anemia | 0.390 | 1.477 | 0.891 | 2.449 | 0.13 |
Comorbidities . | Coefficient . | OR . | 95% CI for OR . | P-value . | |
---|---|---|---|---|---|
. | . | . | Lower . | Upper . | . |
pH ≤ 7.30 | 0.977 | 2.656 | 1.436 | 4.916 | 0.002 |
Nephrotoxin exposure | 0.929 | 2.532 | 1.497 | 4.281 | 0.001 |
Chronic kidney disease | 0.860 | 2.363 | 1.153 | 4.842 | 0.02 |
Chronic liver disease | 0.778 | 2.177 | 1.118 | 4.239 | 0.02 |
Severe infection/sepsis | 0.743 | 2.102 | 1.175 | 3.763 | 0.01 |
Congestive heart failure | 0.720 | 2.054 | 1.059 | 3.985 | 0.03 |
Hypertension | 0.563 | 1.756 | 1.084 | 2.844 | 0.02 |
Atherosclerotic coronary vascular disease | 0.490 | 1.632 | 0.905 | 2.945 | 0.10 |
Mechanical ventilation | 0.447 | 1.564 | 0.981 | 2.493 | 0.06 |
Anemia | 0.390 | 1.477 | 0.891 | 2.449 | 0.13 |
OR, odds ratio.
Predictors of AKI obtained by binary multivariate logistic regression analysis in the UCSD development cohort (n = 573 patients)
Comorbidities . | Coefficient . | OR . | 95% CI for OR . | P-value . | |
---|---|---|---|---|---|
. | . | . | Lower . | Upper . | . |
pH ≤ 7.30 | 0.977 | 2.656 | 1.436 | 4.916 | 0.002 |
Nephrotoxin exposure | 0.929 | 2.532 | 1.497 | 4.281 | 0.001 |
Chronic kidney disease | 0.860 | 2.363 | 1.153 | 4.842 | 0.02 |
Chronic liver disease | 0.778 | 2.177 | 1.118 | 4.239 | 0.02 |
Severe infection/sepsis | 0.743 | 2.102 | 1.175 | 3.763 | 0.01 |
Congestive heart failure | 0.720 | 2.054 | 1.059 | 3.985 | 0.03 |
Hypertension | 0.563 | 1.756 | 1.084 | 2.844 | 0.02 |
Atherosclerotic coronary vascular disease | 0.490 | 1.632 | 0.905 | 2.945 | 0.10 |
Mechanical ventilation | 0.447 | 1.564 | 0.981 | 2.493 | 0.06 |
Anemia | 0.390 | 1.477 | 0.891 | 2.449 | 0.13 |
Comorbidities . | Coefficient . | OR . | 95% CI for OR . | P-value . | |
---|---|---|---|---|---|
. | . | . | Lower . | Upper . | . |
pH ≤ 7.30 | 0.977 | 2.656 | 1.436 | 4.916 | 0.002 |
Nephrotoxin exposure | 0.929 | 2.532 | 1.497 | 4.281 | 0.001 |
Chronic kidney disease | 0.860 | 2.363 | 1.153 | 4.842 | 0.02 |
Chronic liver disease | 0.778 | 2.177 | 1.118 | 4.239 | 0.02 |
Severe infection/sepsis | 0.743 | 2.102 | 1.175 | 3.763 | 0.01 |
Congestive heart failure | 0.720 | 2.054 | 1.059 | 3.985 | 0.03 |
Hypertension | 0.563 | 1.756 | 1.084 | 2.844 | 0.02 |
Atherosclerotic coronary vascular disease | 0.490 | 1.632 | 0.905 | 2.945 | 0.10 |
Mechanical ventilation | 0.447 | 1.564 | 0.981 | 2.493 | 0.06 |
Anemia | 0.390 | 1.477 | 0.891 | 2.449 | 0.13 |
OR, odds ratio.
. | Risk factor . | Points . |
---|---|---|
Chronic | Chronic kidney disease | 2 |
Chronic liver disease | 2 | |
Congestive heart failure | 2 | |
Hypertension | 2 | |
Atherosclerotic coronary vascular disease | 2 | |
Acute | pH ≤ 7.30 | 3 |
Nephrotoxin exposure | 3 | |
Severe infection/sepsis | 2 | |
Mechanical ventilation | 2 | |
Anemia | 1 |
. | Risk factor . | Points . |
---|---|---|
Chronic | Chronic kidney disease | 2 |
Chronic liver disease | 2 | |
Congestive heart failure | 2 | |
Hypertension | 2 | |
Atherosclerotic coronary vascular disease | 2 | |
Acute | pH ≤ 7.30 | 3 |
Nephrotoxin exposure | 3 | |
Severe infection/sepsis | 2 | |
Mechanical ventilation | 2 | |
Anemia | 1 |
Minimum total score, 0; maximum total score, 21.
. | Risk factor . | Points . |
---|---|---|
Chronic | Chronic kidney disease | 2 |
Chronic liver disease | 2 | |
Congestive heart failure | 2 | |
Hypertension | 2 | |
Atherosclerotic coronary vascular disease | 2 | |
Acute | pH ≤ 7.30 | 3 |
Nephrotoxin exposure | 3 | |
Severe infection/sepsis | 2 | |
Mechanical ventilation | 2 | |
Anemia | 1 |
. | Risk factor . | Points . |
---|---|---|
Chronic | Chronic kidney disease | 2 |
Chronic liver disease | 2 | |
Congestive heart failure | 2 | |
Hypertension | 2 | |
Atherosclerotic coronary vascular disease | 2 | |
Acute | pH ≤ 7.30 | 3 |
Nephrotoxin exposure | 3 | |
Severe infection/sepsis | 2 | |
Mechanical ventilation | 2 | |
Anemia | 1 |
Minimum total score, 0; maximum total score, 21.
Validation of the risk prediction model (Mayo Clinic cohort)

Area under curve of UCSD risk model for prediction of acute kidney injury in Mayo clinic validation cohort.

Calibration curves in UCSD cohort and Mayo clinic validation cohort.
The information on sensitivity, specificity and predictive values according to cutoff points of the score in the UCSD and Mayo Clinic cohorts suggested a threshold of ≥5 points as the optimal cutoff value to define high-risk individuals [sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV) and Youden’s index were 74%, 72%, 23%, 96% and 0.46 and 63%, 85%, 32%, 95% and 0.48, respectively] (Supplementary data, Table E3). The increase in cutoff leads to higher PPVs (data not shown). The cutoff of ≥5 points was chosen based on the best combination of sensitivity and specificity, with the goal to identify high-risk patients as well as avoid electronic alert fatigue.
DISCUSSION
AKI is a major complication of critical illnesses associated with adverse outcomes, increased mortality and significant increases in resource utilization [1–3]. In recent years, standardized diagnostic and staging criteria for AKI have contributed to an improved understanding of the incidence and course of AKI in ICU patients; however, there is wide variation in its timely recognition, management and outcomes [2, 3]. Several sensitive and specific urine and serum biomarkers of kidney injury have emerged for the early detection of AKI [31–38]. These novel markers, including insulin-like growth factor-binding protein 7, tissue inhibitor of metalloproteinases-2, kidney injury molecule-1, neutrophil gelatinase–associated lipocalin, interleukin-18, protein C and cystatin C have shown promising results to predict AKI in adult (AUROC ≤0.8) and pediatric (AUROC >0.95) patient populations [27–32, 63–66] but have not been applied in routine clinical care.
Biomarker alone–based strategies are costly and prone to failure because of the clinical heterogeneity displayed by individual patients. A predictive model for a complex disease such as AKI ideally requires a combination of epidemiological, clinical, biological and hereditary factors along with a panel of biomarkers [67]. In this study, we have developed and validated a risk score derived from patient’s demographics, chronic comorbidities and acute risk factors and demonstrated that it can reliably predict AKI in a critically ill adult population at ICU admission, with an AUROC of 0.79 and AUROC of 0.81 in two different cohorts (Figures 2 and 3). The optimal cutoff for the diagnosis of AKI was estimated to be ≥5 points (Supplementary data, Table E3). With this score value, our model identifies 40% of the ICU patient population as high risk, 23% of which will likely develop AKI within 48 h. In contrast, 96% of the individuals with a score <5 may not develop AKI.
The risk factors we used for identification of AKI are consistent with previous AKI literature [30, 39–57]. Several of the AKI predictors used in our study have been identified previously in other AKI risk scores: CHF, HTN, CKD, nephrotoxin exposure, chronic liver disease and sepsis [9–19, 45]. We also observed that acidosis, MV, atherosclerotic coronary artery disease and anemia were predictors of AKI in our model. Anemia and acidosis are potentially modifiable and could be targeted for correction [25, 54].
Our findings also confirm the high incidence of AKI in ICU patients seen in other studies [2–3, 68], with the majority of patients developing AKI within the first 48 h after study enrollment. AKI patients had a higher mortality rate and increased LOS in the ICU, supporting the need for identifying high-risk patients who could benefit from surveillance and primary prevention strategies to reduce the chance for AKI. Although risk factors for AKI have been identified in other settings and are utilized for patient management [10, 14], there are limited risk prediction tools for AKI development in adult ICU patients [24–26, 30]. Recently, the Renal Angina Index (RAI), based on hazard tranches of clinical factors, was described to identify critically ill children with evidence of early kidney injury who are at risk for developing severe AKI [28]. The RAI has been validated in 506 adult ICU populations to predict AKI stages 2 and 3 [27]. While effectively ruling out patients at low risk for severe AKI, the PPV of the RAI for predicting severe AKI was only 16%, compared with a PPV of 23% for our risk model. Our risk stratification clinical model has a higher the AUROC of 0.79 than the AUROC of 0.74 for the RAI clinical model. In addition, a limitation of the RAI is that SCr was used both to identify risk groups and to define AKI and 14 risk factors were evaluated as predictors of AKI without variable selection regression methodology.
We believe our risk score considering underlying comorbidities with acute risk factors equips clinicians with a new tool to identify high-risk patients and implement preventive strategies, e.g. optimization of volume status, drug dosing adjustments and avoidance of potentially nephrotoxic medicines and procedures. The incorporation of risk prediction tools in electronic databases can allow the automatic detection of high-risk patients for surveillance, and integration with biomarkers can further improve diagnostic accuracy and help in the early management and individualization of treatment for AKI, facilitating patient counseling [69–74]. From a research standpoint, we anticipate that the risk profile models will assist in designing more sophisticated and effective clinical trials for AKI to improve patient selection for prevention and intervention studies.
Our study has several strengths. Data were collected prospectively with a predefined standardized definition of the risk variables within the development and validation cohorts (Supplementary data, Table E1). Most of the major risk variables associated with AKI were included in the study. Risk score development was based on parameters available to clinicians. We can estimate AKI risk via the regression equation or an easy-to-use categorical formulation. The patient population was fairly heterogeneous and well representative of critically ill patients with both medical and surgical conditions. The risk model was developed in multicenter cohorts, with a relatively large population size and showed good discrimination despite significant differences in underlying comorbidities and the incidence of AKI. These differences are significant, as previous studies by Coritsidis et al. [24], Peres et al. [25], Chawla et al. [26] and Hoste et al. [30] that examined risk prediction of AKI in the ICU setting were small, homogeneous and single center with no internal or external validation. Also, not all the relevant covariates were evaluated in previous AKI risk prediction models [24–27, 30] (Tables 4 and 5).
Comparison of risk factors in risk prediction studies for AKI in the ICU setting
Risk factors . | Coritsidis et al. [24] . | Hoste et al. [30] . | Chawla et al.[26] . | Renal angina index [28] . | Peres et al. [25] . | UCSD–Mayo model . |
---|---|---|---|---|---|---|
Nephrotoxin exposure | × | × | ||||
Chronic kidney disease | × | × | ||||
Diabetes | × | |||||
Severe infection/sepsis | × | × | × | |||
Congestive heart failure | × | |||||
Cardiovascular disease | × | × | ||||
Hypertension | × | × | ||||
Hypotension | × | |||||
Metabolic acidosis | × (pH ≤ 7.35) | × | ||||
Mechanical ventilation | × | × | ||||
Anemia | × | |||||
SCr on admission | × (≥ 1 mg/dL) | × | ||||
Urea on admission | × | |||||
Serum albumin | × | × | ||||
Urine osmolality | × | |||||
Active cancer | × | × | ||||
A-a gradient | × | |||||
Hyperglycemia | ||||||
Obesity | × | |||||
Age | × | |||||
High-risk surgery | × | |||||
Aids | × | |||||
Cerebrovascular accident | × | |||||
Hyperbilirubinemia | × | |||||
Early SCr elevation | × |
Risk factors . | Coritsidis et al. [24] . | Hoste et al. [30] . | Chawla et al.[26] . | Renal angina index [28] . | Peres et al. [25] . | UCSD–Mayo model . |
---|---|---|---|---|---|---|
Nephrotoxin exposure | × | × | ||||
Chronic kidney disease | × | × | ||||
Diabetes | × | |||||
Severe infection/sepsis | × | × | × | |||
Congestive heart failure | × | |||||
Cardiovascular disease | × | × | ||||
Hypertension | × | × | ||||
Hypotension | × | |||||
Metabolic acidosis | × (pH ≤ 7.35) | × | ||||
Mechanical ventilation | × | × | ||||
Anemia | × | |||||
SCr on admission | × (≥ 1 mg/dL) | × | ||||
Urea on admission | × | |||||
Serum albumin | × | × | ||||
Urine osmolality | × | |||||
Active cancer | × | × | ||||
A-a gradient | × | |||||
Hyperglycemia | ||||||
Obesity | × | |||||
Age | × | |||||
High-risk surgery | × | |||||
Aids | × | |||||
Cerebrovascular accident | × | |||||
Hyperbilirubinemia | × | |||||
Early SCr elevation | × |
× means presence (yes) of the variable.
Comparison of risk factors in risk prediction studies for AKI in the ICU setting
Risk factors . | Coritsidis et al. [24] . | Hoste et al. [30] . | Chawla et al.[26] . | Renal angina index [28] . | Peres et al. [25] . | UCSD–Mayo model . |
---|---|---|---|---|---|---|
Nephrotoxin exposure | × | × | ||||
Chronic kidney disease | × | × | ||||
Diabetes | × | |||||
Severe infection/sepsis | × | × | × | |||
Congestive heart failure | × | |||||
Cardiovascular disease | × | × | ||||
Hypertension | × | × | ||||
Hypotension | × | |||||
Metabolic acidosis | × (pH ≤ 7.35) | × | ||||
Mechanical ventilation | × | × | ||||
Anemia | × | |||||
SCr on admission | × (≥ 1 mg/dL) | × | ||||
Urea on admission | × | |||||
Serum albumin | × | × | ||||
Urine osmolality | × | |||||
Active cancer | × | × | ||||
A-a gradient | × | |||||
Hyperglycemia | ||||||
Obesity | × | |||||
Age | × | |||||
High-risk surgery | × | |||||
Aids | × | |||||
Cerebrovascular accident | × | |||||
Hyperbilirubinemia | × | |||||
Early SCr elevation | × |
Risk factors . | Coritsidis et al. [24] . | Hoste et al. [30] . | Chawla et al.[26] . | Renal angina index [28] . | Peres et al. [25] . | UCSD–Mayo model . |
---|---|---|---|---|---|---|
Nephrotoxin exposure | × | × | ||||
Chronic kidney disease | × | × | ||||
Diabetes | × | |||||
Severe infection/sepsis | × | × | × | |||
Congestive heart failure | × | |||||
Cardiovascular disease | × | × | ||||
Hypertension | × | × | ||||
Hypotension | × | |||||
Metabolic acidosis | × (pH ≤ 7.35) | × | ||||
Mechanical ventilation | × | × | ||||
Anemia | × | |||||
SCr on admission | × (≥ 1 mg/dL) | × | ||||
Urea on admission | × | |||||
Serum albumin | × | × | ||||
Urine osmolality | × | |||||
Active cancer | × | × | ||||
A-a gradient | × | |||||
Hyperglycemia | ||||||
Obesity | × | |||||
Age | × | |||||
High-risk surgery | × | |||||
Aids | × | |||||
Cerebrovascular accident | × | |||||
Hyperbilirubinemia | × | |||||
Early SCr elevation | × |
× means presence (yes) of the variable.
Characteristics . | Coritsidis et al. [24] . | Hoste et al. [30] . | Chawla et al.[26] . | Renal angina index [28] . | Peres et al. [25] . | UCSD–Mayo model . |
---|---|---|---|---|---|---|
Year of publication | 1995 | 2003 | 2005 | 2014 | 2015 | |
Years data acquired | 2002–03 | 2009–10 | 2012–13 | 2006–08, 2010 | ||
Study cohort | n = 115 | n =185 (sepsis) | n = 194 | n = 506 | n = 152 | n = 717, n = 1300 |
Prospective | Retrospective | Prospective | Prospective (secondary analysis) | Retrospective | Prospective | |
Single center | Single center | Single center | Multicenter | Single center | Multicenter | |
Input variables | Clinical, laboratory, procedural | Clinical, laboratory, procedural | Clinical, laboratory, procedural | Clinical, laboratory, procedural | Clinical, laboratory, procedural | Clinical, laboratory, procedural |
Primary outcome variable | ARF (SCr ≥ 0.5 mg/dL) | ARF (increase from normal SCr to at least SCr ≥ 2) | ARF (>75% increase in SCr if baseline creatinine ≤2.0, or >50% increase in SCr if baseline creatinine >2.0) | Severe ARF (AKIN stages 2 and 3) | ARF (>0.3 or increases >50%) | ARF (absolute increase in SCr level >0.3 mg/dL in 48 h or ≥50% above the reference value within 7 days) |
Statistical model | MLR/OR | MLR/OR | MLR/OR | MLR/OR | MLR/OR | MLR/OR |
Validation | ||||||
By same group | No | No | No | No | No | Internal/external validation |
Independently by others | No | No | No | No | No | No |
Characteristics . | Coritsidis et al. [24] . | Hoste et al. [30] . | Chawla et al.[26] . | Renal angina index [28] . | Peres et al. [25] . | UCSD–Mayo model . |
---|---|---|---|---|---|---|
Year of publication | 1995 | 2003 | 2005 | 2014 | 2015 | |
Years data acquired | 2002–03 | 2009–10 | 2012–13 | 2006–08, 2010 | ||
Study cohort | n = 115 | n =185 (sepsis) | n = 194 | n = 506 | n = 152 | n = 717, n = 1300 |
Prospective | Retrospective | Prospective | Prospective (secondary analysis) | Retrospective | Prospective | |
Single center | Single center | Single center | Multicenter | Single center | Multicenter | |
Input variables | Clinical, laboratory, procedural | Clinical, laboratory, procedural | Clinical, laboratory, procedural | Clinical, laboratory, procedural | Clinical, laboratory, procedural | Clinical, laboratory, procedural |
Primary outcome variable | ARF (SCr ≥ 0.5 mg/dL) | ARF (increase from normal SCr to at least SCr ≥ 2) | ARF (>75% increase in SCr if baseline creatinine ≤2.0, or >50% increase in SCr if baseline creatinine >2.0) | Severe ARF (AKIN stages 2 and 3) | ARF (>0.3 or increases >50%) | ARF (absolute increase in SCr level >0.3 mg/dL in 48 h or ≥50% above the reference value within 7 days) |
Statistical model | MLR/OR | MLR/OR | MLR/OR | MLR/OR | MLR/OR | MLR/OR |
Validation | ||||||
By same group | No | No | No | No | No | Internal/external validation |
Independently by others | No | No | No | No | No | No |
AKIN, Acute Kidney Injury Network; ARF, acute renal failure; MLR, multivariate logistic regression; OR, odds ratio.
Characteristics . | Coritsidis et al. [24] . | Hoste et al. [30] . | Chawla et al.[26] . | Renal angina index [28] . | Peres et al. [25] . | UCSD–Mayo model . |
---|---|---|---|---|---|---|
Year of publication | 1995 | 2003 | 2005 | 2014 | 2015 | |
Years data acquired | 2002–03 | 2009–10 | 2012–13 | 2006–08, 2010 | ||
Study cohort | n = 115 | n =185 (sepsis) | n = 194 | n = 506 | n = 152 | n = 717, n = 1300 |
Prospective | Retrospective | Prospective | Prospective (secondary analysis) | Retrospective | Prospective | |
Single center | Single center | Single center | Multicenter | Single center | Multicenter | |
Input variables | Clinical, laboratory, procedural | Clinical, laboratory, procedural | Clinical, laboratory, procedural | Clinical, laboratory, procedural | Clinical, laboratory, procedural | Clinical, laboratory, procedural |
Primary outcome variable | ARF (SCr ≥ 0.5 mg/dL) | ARF (increase from normal SCr to at least SCr ≥ 2) | ARF (>75% increase in SCr if baseline creatinine ≤2.0, or >50% increase in SCr if baseline creatinine >2.0) | Severe ARF (AKIN stages 2 and 3) | ARF (>0.3 or increases >50%) | ARF (absolute increase in SCr level >0.3 mg/dL in 48 h or ≥50% above the reference value within 7 days) |
Statistical model | MLR/OR | MLR/OR | MLR/OR | MLR/OR | MLR/OR | MLR/OR |
Validation | ||||||
By same group | No | No | No | No | No | Internal/external validation |
Independently by others | No | No | No | No | No | No |
Characteristics . | Coritsidis et al. [24] . | Hoste et al. [30] . | Chawla et al.[26] . | Renal angina index [28] . | Peres et al. [25] . | UCSD–Mayo model . |
---|---|---|---|---|---|---|
Year of publication | 1995 | 2003 | 2005 | 2014 | 2015 | |
Years data acquired | 2002–03 | 2009–10 | 2012–13 | 2006–08, 2010 | ||
Study cohort | n = 115 | n =185 (sepsis) | n = 194 | n = 506 | n = 152 | n = 717, n = 1300 |
Prospective | Retrospective | Prospective | Prospective (secondary analysis) | Retrospective | Prospective | |
Single center | Single center | Single center | Multicenter | Single center | Multicenter | |
Input variables | Clinical, laboratory, procedural | Clinical, laboratory, procedural | Clinical, laboratory, procedural | Clinical, laboratory, procedural | Clinical, laboratory, procedural | Clinical, laboratory, procedural |
Primary outcome variable | ARF (SCr ≥ 0.5 mg/dL) | ARF (increase from normal SCr to at least SCr ≥ 2) | ARF (>75% increase in SCr if baseline creatinine ≤2.0, or >50% increase in SCr if baseline creatinine >2.0) | Severe ARF (AKIN stages 2 and 3) | ARF (>0.3 or increases >50%) | ARF (absolute increase in SCr level >0.3 mg/dL in 48 h or ≥50% above the reference value within 7 days) |
Statistical model | MLR/OR | MLR/OR | MLR/OR | MLR/OR | MLR/OR | MLR/OR |
Validation | ||||||
By same group | No | No | No | No | No | Internal/external validation |
Independently by others | No | No | No | No | No | No |
AKIN, Acute Kidney Injury Network; ARF, acute renal failure; MLR, multivariate logistic regression; OR, odds ratio.
There are also limitations to our study. First, the urine criterion was not applied to diagnose AKI in the UCSD cohort, as urine output data were not uniformly available, and this may have decreased the overall incidence of AKI diagnosis [68, 75]. Second, we could not determine all the risk variables associated with AKI, including hypovolemia, oliguria and severity of illness scores, as the standard of care labs were applied with variable frequency across the cohorts. Third, baseline SCr values were missing in 24% (n = 172) of ICU patients in the UCSD cohort and 30% (n = 384) in the Mayo cohort. We performed a sensitivity analysis after excluding ICU patients with missing baseline SCr values and the AKI risk score model developed was very similar to the earlier model (data not shown). The patient population of our cohorts was not very sick and the majority of our patients developed AKI within the first 48 h of ICU admission, thus limiting the generalizability of our results. Finally, it was not always possible to ascertain chronic risk factors in unconscious ICU patients when the family was not available to provide a medical history. Further refinements could explore a separate risk scoring system for AKI in patients with unknown baseline risk who are still exposed to acute insults.
CONCLUSION
We developed a simple, reliable risk model using readily available clinical variables that can be used to predict AKI at ICU admission. Implementation of this risk model in clinical practice may help target high-risk patients for surveillance and enable clinicians to evaluate novel diagnostic, preventive and therapeutic modalities to mitigate the devastating consequences of AKI. Future studies are needed to prospectively validate our model to predict AKI in independent datasets with different backgrounds.
SUPPLEMENTARY DATA
Supplementary data are available online at http://ndt.oxfordjournals.org.
ACKNOWLEDGEMENTS
R.Me, R.Ma, K.B.K. and J.K. had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. This work was supported by NLM R01 LM009520 (L.O-M.), NIH P30 DK079337 UAB-UCSD O'Brien Core Center for Acute Kidney Injury Research, and NIH U54 HL108460.
AUTHORS’ CONTRIBUTIONS
Study concept and design: R.Ma, R.Me, L.O-M., K.B.K. and E.M.
Acquisition, analysis or interpretation of data: R.Ma, R.Me, L.O-M., K.B.K., E.M., J.K. and S.W.
Drafting of the manuscript: R.Me, R.Ma, L.O-M., K.B.K., E.M. and J.B.
Critical revision of the manuscript for intellectual content: R.Me, L.O-M., K.B.K., J.B. and G.L.
Statistical analysis: J.K., K.B.K., R.Ma and E.M.
Obtained funding: R.Me, L.O-M. and K.B.K.
Administrative, technical or material support: S.W.
Study supervision: R.Me, K.B.K. and L.O-M.
CONFLICT OF INTEREST STATEMENT
None declared.
REFERENCES
- anemia
- primary prevention
- hypertension
- coronary arteriosclerosis
- sepsis
- kidney diseases
- kidney failure, chronic
- congestive heart failure
- liver disease, chronic
- renal failure, acute
- calibration
- comorbidity
- critical illness
- intensive care unit
- risk assessment
- roc curve
- mortality
- mechanical ventilation
- surveillance, medical
- creatinine tests, serum
- predictor variable
- therapeutic intervention
- early diagnosis
- prevention
- area under the roc curve
Comments