Abstract

With advances in the effectiveness of treatment and disease management, the contribution of chronic comorbid diseases (comorbidities) found within the Charlson comorbidity index to mortality is likely to have changed since development of the index in 1984. The authors reevaluated the Charlson index and reassigned weights to each condition by identifying and following patients to observe mortality within 1 year after hospital discharge. They applied the updated index and weights to hospital discharge data from 6 countries and tested for their ability to predict in-hospital mortality. Compared with the original Charlson weights, weights generated from the Calgary, Alberta, Canada, data (2004) were 0 for 5 comorbidities, decreased for 3 comorbidities, increased for 4 comorbidities, and did not change for 5 comorbidities. The C statistics for discriminating in-hospital mortality between the new score generated from the 12 comorbidities and the Charlson score were 0.825 (new) and 0.808 (old), respectively, in Australian data (2008), 0.828 and 0.825 in Canadian data (2008), 0.878 and 0.882 in French data (2004), 0.727 and 0.723 in Japanese data (2008), 0.831 and 0.836 in New Zealand data (2008), and 0.869 and 0.876 in Swiss data (2008). The updated index of 12 comorbidities showed good-to-excellent discrimination in predicting in-hospital mortality in data from 6 countries and may be more appropriate for use with more recent administrative data.

The Charlson comorbidity index (1), a method of predicting mortality by classifying or weighting comorbid conditions (comorbidities), has been widely utilized by health researchers to measure burden of disease and case mix. Since the publication of Charlson et al.’s original article in 1987 (1), the paper has been cited nearly 5,500 times, and the index has been validated for its ability to predict mortality in various disease subgroups, including cancer, renal disease, stroke, intensive care, and liver disease (2–8). These studies consistently demonstrate that the Charlson index is a valid prognostic indicator for mortality.

In 1984, Charlson et al. defined the clinical conditions to be included in the index after a review of 559 hospital charts for patients admitted to medical services at 1 hospital and then assessed the association of these comorbidities with 1-year all-cause mortality (1). Among many potential comorbidity variables assessed, 17 were found to be associated with 1-year mortality. To measure disease burden, Charlson et al. assigned a weighted score to each comorbid condition based on the relative risk of 1-year mortality. After validating the index in breast cancer patients, Charlson et al. reported that the score as an indicator of disease burden had a strong ability to predict mortality (1). To apply the index in administrative hospital discharge data, Deyo et al. (9), Romano et al. (10), and D'Hoore et al. (11) independently developed coding algorithms using the International Classification of Diseases, Ninth Revision (ICD-9), and its clinical modification (ICD-9-CM). Later, Quan et al. (12) developed International Classification of Diseases, Tenth Revision (ICD-10), coding algorithms to define the Charlson index, and Sundararajan et al. (13) assessed the index's performance in ICD-10 international hospital discharge abstract databases.

With advances in chronic disease management and improvements in treatments and technology, patients now survive longer than they did in 1984 when the original Charlson weights were developed. As such, we felt that it was time to reevaluate the Charlson comorbidities and weights for use with more recent data. Therefore, we followed patients discharged from hospitals for 1 year, using hospital and death certification data to reassess the association of the 17 original Charlson comorbidities with all-cause mortality. We then validated the updated comorbidities and score in hospital discharge abstract data from 6 countries.

MATERIALS AND METHODS

Testing population

The study population included patients who were admitted to hospitals in the Calgary Health Region (Alberta, Canada) in 2004 (population 1.3 million (14)). Canada has a government-financed universal health insurance system. Health-record coders review patient charts and code up to 25 diagnoses and up to 20 procedures using the Canadian version of the ICD-10 and the Canadian Classification of Interventions in the hospital discharge abstracts. For each diagnosis code, a 1-digit “diagnosis type” code is assigned to specify the timing of diagnosis (the principal diagnosis is the diagnosis primarily responsible for resource use). Conditions that arise or are diagnosed after hospital admission are labeled postadmission comorbidities or complications.

The analysis sample included Alberta residents aged 18 years and over. For patients with more than 1 hospital admission, we selected the last admission within the year, using the unique personal health number that was recorded in the data.

Defining Charlson comorbidities and outcomes

Using the ICD-10 coding algorithm developed by Quan et al. (12), we identified the Charlson comorbidities in any of the secondary diagnosis coding fields, excluding conditions that occurred or were diagnosed during hospitalization on the basis of the diagnosis type indicator. For each patient, we retrieved all records 1 year prior to the date of the index hospitalization to identify comorbidities. We defined comorbidities in the previous admissions using major and secondary diagnoses, without consideration of diagnosis type. The presence of a comorbid condition was assigned to a patient when it was present in index or previous admission records. Otherwise, the absence of the condition was assigned to the patient.

The outcomes were all-cause mortality in hospital, at 30 days, and 1 year after admission. In-hospital mortality was defined as death recorded in the hospital discharge data. To determine mortality after discharge, we linked records with the Alberta vital statistics registry using common identifiers of personal health number, name, sex, and date of birth between the 2 databases (15). The vital statistics registry captures nearly all deaths that occur in the province.

External validation

The performance of the updated Charlson index and score was assessed in hospital discharge data from Australia (Victoria State data), Canada (national data), France (national data), Japan (national data), New Zealand (national data), and Switzerland (national data). Comorbidities were defined using secondary diagnoses (excluding major/most responsible diagnosis) in each discharge record for patients aged ≥18 years. We excluded conditions that arose or were diagnosed during hospitalization using diagnosis type indicators in the Canadian databases. For persons with multiple hospital admissions, only the first admission was included in Australia data.

Statistical analysis and weight assignment

We calculated the frequencies of comorbidities in the testing population. A Cox proportional hazards model was fitted, with mortality within 1 year following admission as the dependent variable and age, sex, and individual comorbidities as independent variables. Collinearity between comorbidities was assessed using a stepwise method. Age, sex, and 1 comorbidity were added to the model as independent variables to assess the relation between comorbidity and mortality. Then 2 comorbidities were added to the model, and changes in the coefficient for comorbidities after addition of the second comorbidity were observed. We repeated this modeling method by taking out the previously added variable and adding another comorbidity variable until the relation of comorbidities and each of the remaining independent variables had been examined. Using the hazard ratios from this model, we updated the weights for the revised Charlson index (1) in the following manner: a weight of 1 for comorbidities with a risk- adjusted hazard ratio of ≥1.2 but <1.5, a weight of 2 for a hazard ratio of ≥1.5 but <2.5, a weight of 3 for a hazard ratio of ≥2.5 but <3.5, a weight of 4 for a hazard ratio of ≥3.5 but <4.5, and a weight of 6 for a hazard ratio of ≥6 (no comorbidities had a hazard ratio of 4.5 but <6).

We evaluated the updated Charlson indexes/scores and the revised list of comorbidities in 6 external databases by fitting logistic regression models with in-hospital mortality as the dependent variable and age, sex, and either the individual comorbidities (original or revised) or the Charlson comorbidities/scores (original/revised) as independent variables. We calculated C statistics to assess the discrimination of each model (16). The C statistic is a summary measure of discrimination which quantifies the ability of the model to assign a high probability of mortality to those patients who died. C statistics are equivalent to the area under the receiver operating characteristic curve. C statistics range from 0.5 to 1.0; a measure of 0.5 indicates that the discrimination is caused by chance alone, and 1.0 indicates perfect discrimination.

RESULTS

Testing population

Of 55,929 patients, 35.4% were males and 29.9% were aged 65 years or older (see Table 1). The prevalence of comorbidities ranged from 0.06% for acquired immunodeficiency syndrome (AIDS)/human immunodeficiency virus (HIV) to 7.1% for chronic pulmonary diseases. The 30-day and 1-year mortality were 3.6% and 5.7%, respectively.

Table 1.

Frequency of Independent and Dependent Variables Among 55,929 Patients Aged ≥18 Years Who Were Discharged From Hospitals in Calgary, Alberta, Canada, 2004

Variable 
Male sex 35.4 
Age ≥65 years 29.9 
Charlson comorbidity  
    Myocardial infarction 5.0 
    Congestive heart failure 5.0 
    Peripheral vascular disease 2.4 
    Cerebrovascular disease 3.0 
    Hemiplegia or paraplegia 1.4 
    Dementia 3.2 
    Chronic pulmonary disease 7.1 
    Rheumatologic disease 1.2 
    Peptic ulcer disease 1.1 
    Diabetes without chronic complications 6.9 
    Diabetes with chronic complications 1.9 
    Renal disease 3.6 
    Any malignancy, including leukemia and lymphoma 5.0 
    Metastatic solid tumor 3.2 
    Mild liver disease 1.0 
    Moderate or severe liver disease 0.5 
    AIDS/HIV 0.06 
Mortality  
    In hospital 3.5 
    30 days 3.6 
    1 year 5.7 
Variable 
Male sex 35.4 
Age ≥65 years 29.9 
Charlson comorbidity  
    Myocardial infarction 5.0 
    Congestive heart failure 5.0 
    Peripheral vascular disease 2.4 
    Cerebrovascular disease 3.0 
    Hemiplegia or paraplegia 1.4 
    Dementia 3.2 
    Chronic pulmonary disease 7.1 
    Rheumatologic disease 1.2 
    Peptic ulcer disease 1.1 
    Diabetes without chronic complications 6.9 
    Diabetes with chronic complications 1.9 
    Renal disease 3.6 
    Any malignancy, including leukemia and lymphoma 5.0 
    Metastatic solid tumor 3.2 
    Mild liver disease 1.0 
    Moderate or severe liver disease 0.5 
    AIDS/HIV 0.06 
Mortality  
    In hospital 3.5 
    30 days 3.6 
    1 year 5.7 

Abbreviations: AIDS, acquired immunodeficiency syndrome; HIV, human immunodeficiency virus.

Of the 17 comorbidities, 5 were not associated with mortality within the 1-year follow-up period and were assigned a weight of 0: myocardial infarction, peripheral vascular disease, cerebrovascular disease, peptic ulcer disease, and diabetes without chronic complications (see Table 2). Compared with the original Charlson weights, the updated weights increased for congestive heart failure, dementia, mild liver disease, and moderate or severe liver disease; decreased for diabetes with chronic complications, renal disease, and AIDS/HIV; and were unchanged for chronic pulmonary disease, rheumatologic disease, hemiplegia or paraplegia, any malignancy, and metastatic solid tumor. The maximum score for a patient was 24 according to the updated scoring method as compared with 29 for the Charlson index.

Table 2.

Risk-Adjusted Hazard Ratio for Mortality Within 1 Year After Hospital Discharge Among 55,929 Patients Aged ≥18 Years, Calgary, Alberta, Canada, 2004

Variable Hazard Ratio Updated Weight Charlson Weight 
Male sex 1.28   
Age ≥65 years 4.40   
Charlson comorbiditya    
    Myocardial infarction 0.99* 
    Congestive heart failure 1.91 
    Peripheral vascular disease 1.10* 
    Cerebrovascular disease 1.10* 
    Dementia 2.39 
    Chronic pulmonary disease 1.28 
    Rheumatologic disease 1.30 
    Peptic ulcer disease 1.08* 
    Mild liver disease 1.94 
    Diabetes without chronic complications 1.12* 
    Diabetes with chronic complications 1.22 
    Hemiplegia or paraplegia 2.26 
    Renal disease 1.43 
    Any malignancy, including leukemia and lymphoma 2.28 
    Moderate or severe liver disease 3.83 
    Metastatic solid tumor 6.01 
    AIDS/HIV 3.69 
Maximum comorbidity score  24 29 
Variable Hazard Ratio Updated Weight Charlson Weight 
Male sex 1.28   
Age ≥65 years 4.40   
Charlson comorbiditya    
    Myocardial infarction 0.99* 
    Congestive heart failure 1.91 
    Peripheral vascular disease 1.10* 
    Cerebrovascular disease 1.10* 
    Dementia 2.39 
    Chronic pulmonary disease 1.28 
    Rheumatologic disease 1.30 
    Peptic ulcer disease 1.08* 
    Mild liver disease 1.94 
    Diabetes without chronic complications 1.12* 
    Diabetes with chronic complications 1.22 
    Hemiplegia or paraplegia 2.26 
    Renal disease 1.43 
    Any malignancy, including leukemia and lymphoma 2.28 
    Moderate or severe liver disease 3.83 
    Metastatic solid tumor 6.01 
    AIDS/HIV 3.69 
Maximum comorbidity score  24 29 

Abbreviations: AIDS, acquired immunodeficiency syndrome; HIV, human immunodeficiency virus.

*

P > 0.05.

a

The following comorbid conditions were mutually exclusive: diabetes with chronic complications and diabetes without chronic complications; mild liver disease and moderate or severe liver disease; and any malignancy and metastatic solid tumor.

The C statistics (see Table 3) for the relation between the updated index and the original Charlson index were similar for the logistic models predicting in-hospital, 30-day, and 1-year mortality, regardless of whether individual comorbidities or scores were used.

Table 3.

C Statistic From Risk-Adjusted Logistic Regression Analysis Predicting Mortality Among 55,929 Patients Aged ≥18 Years, Calgary, Alberta, Canada, 2004

Mortality Measure C Statistica
 
12 Updated Comorbidities 17 Charlson Comorbidities Updated Score Charlson Score 
In hospital 0.882 0.884 0.881 0.879 
At 30 days 0.884 0.886 0.883 0.881 
At 1 year 0.897 0.899 0.896 0.894 
Mortality Measure C Statistica
 
12 Updated Comorbidities 17 Charlson Comorbidities Updated Score Charlson Score 
In hospital 0.882 0.884 0.881 0.879 
At 30 days 0.884 0.886 0.883 0.881 
At 1 year 0.897 0.899 0.896 0.894 
a

Independent variables included male sex, age ≥65 years, and individual comorbidities/comorbidity score. The dependent variable was mortality.

External validation

Among the 6 hospital samples, the proportion of patients aged 65 years or older was highest in Japan (56.9%) and lowest in New Zealand (37.2%) (see Table 4). The proportion of male patients ranged from 40.7% to 58.3%. Prevalences of the 12 updated comorbidities ranged from 19.1% to 31.9%, and in-hospital mortality ranged from 1.4% to 4.4%.

Table 4.

Characteristics of the External Study Population and C Statistics for Models Predicting In-Hospital Mortality Among Patients Aged ≥18 Years in 6 Countries, 2004 and 2008

 Australia (2008)
 
Canada (2008)
 
France (2004)
 
Japan (2008)
 
New Zealand (2008)
 
Switzerland (2008)
 
 No. or % C No. or % C No. or % C No. or % C No. or % C No. or % C 
Characteristic             
    No. of persons 352,200  1,894,843  779,336  2,361,957  670,908  788,355  
    Age ≥65 years, % 47.5  41.6  33.6  56.9  37.2  37.4  
    Male sex, % 57.4  40.7  42.5  53.3  58.3  43.3  
    Presence of 12 updated comorbidities, % 31.9  25.7  19.1  27.9  20.8  19.2  
    In-hospital mortality, % 3.5  4.4  2.6  4.1  1.4  3.0  
Independent variables             
    Age, sex, and 12 updated comorbidities  0.833  0.829  0.883  0.731  0.838  0.874 
    Age, sex, and 17 Charlson comorbidities  0.837  0.835  0.895  0.736  0.848  0.887 
    Age, sex, and score from the 12 updated comorbidities  0.825  0.828  0.878  0.727  0.831  0.869 
    Age, sex, and score from the 17 Charlson comorbidities  0.808  0.825  0.882  0.723  0.836  0.876 
 Australia (2008)
 
Canada (2008)
 
France (2004)
 
Japan (2008)
 
New Zealand (2008)
 
Switzerland (2008)
 
 No. or % C No. or % C No. or % C No. or % C No. or % C No. or % C 
Characteristic             
    No. of persons 352,200  1,894,843  779,336  2,361,957  670,908  788,355  
    Age ≥65 years, % 47.5  41.6  33.6  56.9  37.2  37.4  
    Male sex, % 57.4  40.7  42.5  53.3  58.3  43.3  
    Presence of 12 updated comorbidities, % 31.9  25.7  19.1  27.9  20.8  19.2  
    In-hospital mortality, % 3.5  4.4  2.6  4.1  1.4  3.0  
Independent variables             
    Age, sex, and 12 updated comorbidities  0.833  0.829  0.883  0.731  0.838  0.874 
    Age, sex, and 17 Charlson comorbidities  0.837  0.835  0.895  0.736  0.848  0.887 
    Age, sex, and score from the 12 updated comorbidities  0.825  0.828  0.878  0.727  0.831  0.869 
    Age, sex, and score from the 17 Charlson comorbidities  0.808  0.825  0.882  0.723  0.836  0.876 

The C statistic in a model with age, sex, and the 12 updated individual comorbidities for each sample ranged from 0.731 to 0.883. The ability of the model to discriminate in-hospital mortality was similar between the updated index and the original Charlson index in each of the 6 external databases (see Table 4). The model with individual comorbidities had a slightly higher C statistic than the model with a summarized score (for example, 0.883 for the model with 12 comorbidities versus 0.878 for the model with a score generated from these 12 comorbidities in French data). The in-hospital mortality increased linearly with the updated score (see Table 5).

Table 5.

In-Hospital Mortality (%) by Comorbidity Score Among Patients Aged ≥18 Years in 6 Countries, 2004 and 2008

Country Comorbidity Score
 
≥6 
Australia (2008)        
    Updated score 1.2 3.1 6.8 9.2 13.6 18.3 20.0 
    Charlson score 1.0 4.6 4.9 12.9 9.9 12.5 18.8 
Canada (2008)        
    Updated score 1.7 6.1 11.1 14.9 19.0 23.6 24.7 
    Charlson score 1.5 6.0 8.5 12.0 13.4 17.1 24.8 
France (2004)        
    Updated score 0.9 2.4 8.0 13.5 15.4 22.7 31.9 
    Charlson score 0.6 4.4 6.4 12.6 16.4 22.3 28.3 
Japan (2008)        
    Updated score 2.6 4.6 6.9 9.9 13.4 17.8 20.0 
    Charlson score 2.4 4.4 5.9 8.6 9.0 16.1 19.6 
New Zealand (2008)        
    Updated score 0.5 2.2 4.5 10.0 10.3 16.6 7.8 
    Charlson score 0.4 2.9 3.0 7.2 5.7 8.2 8.0 
Switzerland (2008)        
    Updated score 0.9 4.6 8.7 15.2 18.4 21.7 31.4 
    Charlson score 0.7 4.4 7.6 12.4 16.1 18.6 31.2 
Country Comorbidity Score
 
≥6 
Australia (2008)        
    Updated score 1.2 3.1 6.8 9.2 13.6 18.3 20.0 
    Charlson score 1.0 4.6 4.9 12.9 9.9 12.5 18.8 
Canada (2008)        
    Updated score 1.7 6.1 11.1 14.9 19.0 23.6 24.7 
    Charlson score 1.5 6.0 8.5 12.0 13.4 17.1 24.8 
France (2004)        
    Updated score 0.9 2.4 8.0 13.5 15.4 22.7 31.9 
    Charlson score 0.6 4.4 6.4 12.6 16.4 22.3 28.3 
Japan (2008)        
    Updated score 2.6 4.6 6.9 9.9 13.4 17.8 20.0 
    Charlson score 2.4 4.4 5.9 8.6 9.0 16.1 19.6 
New Zealand (2008)        
    Updated score 0.5 2.2 4.5 10.0 10.3 16.6 7.8 
    Charlson score 0.4 2.9 3.0 7.2 5.7 8.2 8.0 
Switzerland (2008)        
    Updated score 0.9 4.6 8.7 15.2 18.4 21.7 31.4 
    Charlson score 0.7 4.4 7.6 12.4 16.1 18.6 31.2 

DISCUSSION

Our study updated the Charlson comorbidity index based on the hazard ratios of individual comorbidities for mortality within 1 year after hospital admission. Only 12 comorbidities were retained in the updated index (as compared with 17 conditions in the original Charlson index). The updated index/score discriminated mortality well in the testing population and 6 validating external databases.

Our findings have 2 major implications. The first implication regards the choice of comorbidities in a study. Comorbidity data are frequently collected through chart review, survey, or registries (17–19). Shortening the list of comorbidities without significantly sacrificing the discrimination of the weighted index saves resources in data collection. The second implication regards data analysis. In risk adjustment, commonly either individual comorbidities are included as independent variables or a score is included. For a study with a small sample size, rare comorbidities (such as AIDS/HIV) may cause instability in model performance when included in the model as dummy variables. The reason for this is that the frequency of the outcome (such as mortality) may be zero among patients with rare conditions.

The updated weight was lower than the Charlson weight for diabetes with chronic complications, renal disease, and AIDS/HIV but higher for congestive heart failure, dementia, mild liver disease, and moderate or severe liver disease. The increase in weight for these comorbidities may be related to an aging population and the increasing severity of disease in hospitalized patients (20–23).

Our analysis demonstrated that myocardial infarction, diabetes without chronic complications, peripheral vascular disease, peptic ulcer disease, and cerebrovascular disease were not associated with mortality within 1 year after hospital admission. The findings are consistent with reports from previous studies. Elixhauser et al. (24) analyzed a large hospital discharge database to assess the association of numerous comorbidities (including Charlson comorbidities) with in-hospital mortality and formed an index. The index did not include myocardial infarction, peptic ulcer disease, or cerebrovascular disease. van Walraven et al. (25) reassessed the Elixhauser comorbidities using 1996 and 2008 hospital discharge abstract administrative data and found that diabetes without chronic complications was not associated with in-hospital mortality whereas peripheral vascular disease was (risk-adjusted odds ratio = 1.26).

We excluded conditions from our analysis considered to be either the diagnosis most responsible for the hospital admission or the one mainly responsible for resource utilization (i.e., the major diagnosis). We also excluded conditions that arose or were diagnosed during hospitalization in defining our comorbidities. This approach may have excluded acute conditions, such as acute myocardial infarction and stroke, as well as chronic conditions, such as old myocardial infarction and consequences of stroke (i.e., hemiplegia or paraplegia).

Comorbidities should include conditions that are present upon admission. In data from some countries (such as Canada, the United States, and Australia), the timing of condition occurrence or diagnosis is flagged. Among 5 disease cohorts and 3 procedure cohorts in New York and California administrative data, Pine et al. (26) compared prediction of in-hospital mortality performance with or without exclusion of conditions present upon admission. The average C statistic increased from 0.79 in the model without consideration of conditions present on admission to 0.84 in the model including conditions present on admission. Ghali et al. (27) ranked in-hospital mortality associated with coronary artery bypass graft surgery among 23 Canadian hospitals using 2 risk adjustment methods (i.e., including or excluding conditions that occurred or were diagnosed in the hospital). The study demonstrated that the hospital rank was not consistent between the 2 analyses and recommended the use of diagnosis type or presence on admission indicators in future risk adjustment analyses.

We fitted logistic regression models using 2 methods of adding independent variables. The models with 12 dummy comorbidity variables (i.e., individual comorbidities) had a slightly higher C statistic than the model with score as the independent variable. Although the C statistic value is related to the number of independent variables (i.e., the more variables the higher the C statistic) (16), we recommend that researchers fit a model with 12 individual conditions rather than the score alone, because we believe this approach enhances the ability of risk adjustment to control for potential confounding.

We found that the C statistic value ranged from 0.727 to 0.878 in the 6 external databases. This variation may reflect the health status or case mix of the hospital populations studied and/or data quality. The purpose of risk adjustment as conceptualized by Iezzoni et al. (28) is to isolate the effects of the intrinsic patient-related risk factors from any assessments of the quality of care. Data errors may occur in the process of creating administrative data due to physician misdiagnosis, incomplete documentation of clinical information in hospital charts, incomplete diagnoses or coders’ miscoding of diagnoses, and the nature of the health-care funding system (29). Considering the potential for overcoding for reimbursement, Hsia et al. (30, 31) assessed the accuracy of US claims data by grouping clinically interrelated diagnostic codes into Diagnosis Related Groups to measure the effect of incorrect coding on Diagnosis Related Group assignment. They reported that coding errors decreased significantly, from 21% in 1985 to 15% in 1988 (30, 31).

Validation studies of Charlson comorbidities that have employed administrative data (18, 32–36) have found that these comorbidity variables were coded reasonably well. Many other investigators (37–52) have conducted validation studies focusing on certain clinical conditions or complications of substandard care and have found that administrative data are accurately coded for many severe or life-threatening conditions, such as cancer, but some clinically nonspecific and symptomatic conditions, such as rheumatologic disease, are less accurately coded. Administrative data may also underestimate the presence of conditions. Japanese data record up to 10 diagnoses, while the other 5 countries’ databases have at least 15 coding fields for secondary diagnoses. Because of the limited number of coding fields for diagnosis, the prevalence of comorbidities identified by the Japanese data is more likely to be an underestimate compared with the results from other countries.

To overcome the potential to underestimate the prevalence of comorbidity, researchers may enhance comorbidity ascertainment by using records of hospital admissions occurring prior to the index admission. Lee et al. (5) defined the prevalence of Charlson comorbidities in both chart and hospital abstract data by using an index admission and a combination of index and prior admissions among heart failure patients. They found that prevalence of comorbidities increased substantially with the inclusion of previous admissions occurring up to 3 years prior to the index admission. Comorbidity prevalence was lower in administrative data than in chart data. In a model predicting 30-day mortality, the C statistic was 0.729 in chart data, 0.691 in index hospital abstract data, 0.694 in abstract data with a combination of the index admission and previous admissions up to 1 year, and 0.703 in abstract data with a combination of the index admission and previous admissions up to 2 years. The C statistic did not increase in the data with a combination of index admission and previous admissions up to 4 years from that of previous admissions up to 3 years. Therefore, enhancing comorbidity prevalence using previous admissions can increase model performance. Because this method is best undertaken with a reliable unique patient identifier to link records across hospitalizations, it is not always possible to maximize comorbidity ascertainment.

A major strength of our study is that patient survival was determined in population-based data. Further, the updated index was validated in external databases from 6 countries in Asia, Europe, the Pacific, and North America. Our findings were consistent across these national databases, indicating that our results are likely to be generalizable to other regions. However, additional validation in disease-specific cohorts and other national data is necessary.

This study had limitations. Charlson et al. defined comorbidities through chart review (1), but we used administrative data. The difference in data quality between these 2 methods may have affected our findings. Quan et al. (36) reviewed 4,008 randomly selected charts for patients admitted to hospitals in Alberta, Canada, in 2003 to assess the agreement between administrative data and chart data in relation to Charlson comorbidities. The kappa value ranged from 0.52 to 0.83, indicating moderate to almost-perfect agreement according to Landis and Koch (53). The second limitation is that clinical severity of disease is not coded in administrative data. Therefore, we could not adjust for that important variable in our survival analysis. The third limitation is that the updated index was developed for predicting 1-year mortality following the method used by Charlson et al. (1). However, the external data validation was conducted by predicting in-hospital mortality, since it was not feasible to determine mortality after discharge in some of these external databases. The fourth limitation is that the model was validated only in developed countries. Generalizability of the method to developing and economically transitional countries requires further evaluation.

In conclusion, our updated Charlson index consists of 12 comorbidities rather than the original 17. The updated index and score show a good ability to discriminate outcome with regard to hospital mortality in 6 developed-country databases but would benefit from validation in other, developing-country databases and disease-/procedure-specific cohorts. Because the Charlson comorbidity index was developed to predict hospital mortality, its performance for predicting health-resource use, such as length of stay, service utilization, and cost, requires further investigation.

Abbreviations

  • AIDS

    acquired immunodeficiency syndrome

  • HIV

    human immunodeficiency virus

  • ICD

    International Classification of Diseases

Author affiliations: Department of Community Health Sciences, Faculty of Medicine, University of Calgary, Calgary, Alberta, Canada (Hude Quan, Bing Li); Hospices Civils de Lyon, Claude Bernard University Lyon I, Lyon, France (Chantal M. Couris); Department of Health Policy, Tokyo Medical and Dental University, Tokyo, Japan (Kiyohide Fushimi); Department of Public Health Care and General Practice, School of Medicine and Health Science, University of Otago, Christchurch, New Zealand (Patrick Graham, Phil Hider); Health Care Evaluation Unit, Institut Universitaire de Médecine Sociale et Préventive, Lausanne, Switzerland (Jean-Marie Januel); and Department of Medicine, Faculty of Medicine, Nursing and Health Sciences, Monash University, Melbourne, Australia (Vijaya Sundararajan).

Dr. Hude Quan was supported by an Alberta Innovates–Health Solutions Health Scholar Award.

Members of the International Methodology Consortium for Coded Health Information (IMECCHI) contributed to the concept of this study. The names of the IMECCHI investigators are listed on the IMECCHI Web site (www.IMECCHI.org).

Conflict of interest: none declared.

References

1.
Charlson
ME
Pompei
P
Ales
KL
, et al.  . 
A new method of classifying prognostic comorbidity in longitudinal studies: development and validation
J Chronic Dis
 , 
1987
, vol. 
40
 
5
(pg. 
373
-
383
)
2.
Baldwin
LM
Klabunde
CN
Green
P
, et al.  . 
In search of the perfect comorbidity measure for use with administrative claims data: does it exist?
Med Care
 , 
2006
, vol. 
44
 
8
(pg. 
745
-
753
)
3.
Goldstein
LB
Samsa
GP
Matchar
DB
, et al.  . 
Charlson index comorbidity adjustment for ischemic stroke outcome studies
Stroke
 , 
2004
, vol. 
35
 
8
(pg. 
1941
-
1945
)
4.
Hemmelgarn
BR
Manns
BJ
Quan
H
, et al.  . 
Adapting the Charlson comorbidity index for use in patients with ESRD
Am J Kidney Dis
 , 
2003
, vol. 
42
 
1
(pg. 
125
-
132
)
5.
Lee
DS
Donovan
L
Austin
PC
, et al.  . 
Comparison of coding of heart failure and comorbidities in administrative and clinical data for use in outcomes research
Med Care
 , 
2005
, vol. 
43
 
2
(pg. 
182
-
188
)
6.
Myers
RP
Quan
H
Hubbard
JN
, et al.  . 
Predicting in-hospital mortality in patients with cirrhosis: results differ across risk adjustment methods
Hepatology
 , 
2009
, vol. 
49
 
2
(pg. 
568
-
577
)
7.
Poses
RM
McClish
DK
Smith
WR
, et al.  . 
Prediction of survival of critically ill patients by admission comorbidity
J Clin Epidemiol
 , 
1996
, vol. 
49
 
7
(pg. 
743
-
747
)
8.
Quach
S
Hennessy
DA
Faris
P
, et al.  . 
A comparison between the APACHE II and Charlson index score for predicting hospital mortality in critically ill patients
BMC Health Serv Res.
 , 
2009
, vol. 
9
 pg. 
129
  
doi: 10.1186/1472-6963-9-129
9.
Deyo
RA
Cherkin
DC
Ciol
MA
Adapting a clinical comorbidity index for use with ICD-9-CM administrative databases
J Clin Epidemiol
 , 
1992
, vol. 
45
 
6
(pg. 
613
-
619
)
10.
Romano
PS
Roos
LL
Jollis
JG
Adapting a clinical comorbidity index for use with ICD-9-CM administrative data: differing perspectives
J Clin Epidemiol
 , 
1993
, vol. 
46
 
10
(pg. 
1075
-
1079
)
11.
D'Hoore
W
Bouckaert
A
Tilquin
C
Practical considerations on the use of the Charlson comorbidity index with administrative data bases
J Clin Epidemiol
 , 
1996
, vol. 
49
 
12
(pg. 
1429
-
1433
)
12.
Quan
H
Sundararajan
V
Halfon
P
, et al.  . 
Coding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data
Med Care
 , 
2005
, vol. 
43
 
11
(pg. 
1130
-
1139
)
13.
Sundararajan
V
Quan
H
Halfon
P
, et al.  . 
Cross-national comparative performance of three versions of the ICD-10 Charlson index
Med Care
 , 
2007
, vol. 
45
 
12
(pg. 
1210
-
1215
)
14.
Alberta Health and Wellness
2007/2008 Regional Health Authority Global Funding Methodology and Funding Manuel
 , 
2007
Edmonton, Alberta, Canada
Alberta Health and Wellness
15.
Li
B
Quan
H
Fong
A
, et al.  . 
Assessing record linkage between health care and vital statistics databases using deterministic methods
BMC Health Serv Res.
 , 
2006
, vol. 
6
 pg. 
48
  
doi: 10.1186/1472-6963-6-48
16.
Cook
NR
Use and misuse of the receiver operating characteristic curve in risk prediction
Circulation
 , 
2007
, vol. 
115
 
7
(pg. 
928
-
935
)
17.
Ghali
WA
Knudtson
ML
Overview of the Alberta Provincial Project for Outcome Assessment in Coronary Heart Disease. On behalf of the APPROACH Investigators
Can J Cardiol
 , 
2000
, vol. 
16
 
10
(pg. 
1225
-
1230
)
18.
Humphries
KH
Rankin
JM
Carere
RG
, et al.  . 
Co-morbidity data in outcomes research: are clinical data derived from administrative databases a reliable alternative to chart review?
J Clin Epidemiol
 , 
2000
, vol. 
53
 
4
(pg. 
343
-
349
)
19.
Chaudhry
S
Jin
L
Meltzer
D
Use of a self-report-generated Charlson Comorbidity Index for predicting mortality
Med Care
 , 
2005
, vol. 
43
 
6
(pg. 
607
-
615
)
20.
Garcin
B
Lillo
P
Hornberger
M
, et al.  . 
Determinants of survival in behavioral variant frontotemporal dementia
Neurology
 , 
2009
, vol. 
73
 
20
(pg. 
1656
-
1661
)
21.
Lee
M
Chodosh
J
Dementia and life expectancy: what do we know?
J Am Med Dir Assoc.
 , 
2009
, vol. 
10
 
7
(pg. 
466
-
471
)
22.
Lefton
HB
Rosa
A
Cohen
M
Diagnosis and epidemiology of cirrhosis
Med Clin North Am.
 , 
2009
, vol. 
93
 
4
(pg. 
787
-
799
vii
23.
Myers
RP
Shaheen
AA
Fong
A
, et al.  . 
Epidemiology and natural history of primary biliary cirrhosis in a Canadian health region: a population-based study
Hepatology
 , 
2009
, vol. 
50
 
6
(pg. 
1884
-
1892
)
24.
Elixhauser
A
Steiner
C
Harris
DR
, et al.  . 
Comorbidity measures for use with administrative data
Med Care
 , 
1998
, vol. 
36
 
1
(pg. 
8
-
27
)
25.
van Walraven
C
Austin
PC
Jennings
A
, et al.  . 
A modification of the Elixhauser comorbidity measures into a point system for hospital death using administrative data
Med Care
 , 
2009
, vol. 
47
 
6
(pg. 
626
-
633
)
26.
Pine
M
Jordan
HS
Elixhauser
A
, et al.  . 
Enhancement of claims data to improve risk adjustment of hospital mortality
JAMA
 , 
2007
, vol. 
297
 
1
(pg. 
71
-
76
)
27.
Ghali
WA
Quan
H
Brant
R
Risk adjustment using administrative data: impact of a diagnosis-type indicator
J Gen Intern Med
 , 
2001
, vol. 
16
 
8
(pg. 
519
-
524
)
28.
Iezzoni
LI
Risk adjustment for medical effectiveness research: an overview of conceptual and methodological considerations
J Investig Med
 , 
1995
, vol. 
43
 
2
(pg. 
136
-
150
)
29.
Schneeweiss
S
Avorn
J
A review of uses of health care utilization databases for epidemiologic research on therapeutics
J Clin Epidemiol
 , 
2005
, vol. 
58
 
4
(pg. 
323
-
337
)
30.
Hsia
DC
Krushat
WM
Fagan
AB
, et al.  . 
Accuracy of diagnostic coding for Medicare patients under the prospective-payment system
N Engl J Med
 , 
1988
, vol. 
318
 
6
(pg. 
352
-
355
)
31.
Hsia
DC
Ahern
CA
Ritchie
BP
, et al.  . 
Medicare reimbursement accuracy under the prospective payment system, 1985 to 1988
JAMA
 , 
1992
, vol. 
268
 
7
(pg. 
896
-
899
)
32.
Kieszak
SM
Flanders
WD
Kosinski
AS
, et al.  . 
A comparison of the Charlson comorbidity index derived from medical record data and administrative billing data
J Clin Epidemiol
 , 
1999
, vol. 
52
 
2
(pg. 
137
-
142
)
33.
Levy
AR
Tamblyn
RM
Fitchett
D
, et al.  . 
Coding accuracy of hospital discharge data for elderly survivors of myocardial infarction
Can J Cardiol
 , 
1999
, vol. 
15
 
11
(pg. 
1277
-
1282
)
34.
Malenka
DJ
McLerran
D
Roos
N
, et al.  . 
Using administrative data to describe casemix: a comparison with the medical record
J Clin Epidemiol
 , 
1994
, vol. 
47
 
9
(pg. 
1027
-
1032
)
35.
Quan
H
Parsons
GA
Ghali
WA
Validity of information on comorbidity derived from ICD-9-CM administrative data
Med Care
 , 
2002
, vol. 
40
 
8
(pg. 
675
-
685
)
36.
Quan
H
Li
B
Saunders
LD
, et al.  . 
Assessing validity of ICD-9-CM and ICD-10 administrative data in recording clinical conditions in a unique dually coded database
Health Serv Res.
 , 
2008
, vol. 
43
 
4
(pg. 
1424
-
1441
)
37.
Benesch
C
Witter
DM
Jr
Wilder
AL
, et al.  . 
Inaccuracy of the International Classification of Diseases (ICD-9-CM) in identifying the diagnosis of ischemic cerebrovascular disease
Neurology
 , 
1997
, vol. 
49
 
3
(pg. 
660
-
664
)
38.
Chen
G
Khan
N
Walker
R
, et al.  . 
Validating ICD coding algorithms for diabetes mellitus from administrative data
Diabetes Res Clin Pract
 , 
2010
, vol. 
89
 
2
(pg. 
189
-
195
)
39.
Hannan
EL
Racz
MJ
Jollis
JG
, et al.  . 
Using Medicare claims data to assess provider quality for CABG surgery: does it work well enough?
Health Serv Res.
 , 
1997
, vol. 
31
 
6
(pg. 
659
-
678
)
40.
Kokotailo
RA
Hill
MD
Coding of stroke and stroke risk factors using International Classification of Diseases, Revisions 9 and 10
Stroke
 , 
2005
, vol. 
36
 
8
(pg. 
1776
-
1781
)
41.
Quan
H
Khan
N
Hemmelgarn
BR
, et al.  . 
Validation of a case definition to define hypertension using administrative data
Hypertension
 , 
2009
, vol. 
54
 
6
(pg. 
1423
-
1428
)
42.
Raiford
DS
Pérez Gutthann
S
García Rodríguez
LA
Positive predictive value of ICD-9 codes in the identification of cases of complicated peptic ulcer disease in the Saskatchewan Hospital automated database
Epidemiology
 , 
1996
, vol. 
7
 
1
(pg. 
101
-
104
)
43.
Rawson
NS
Malcolm
E
Validity of the recording of ischaemic heart disease and chronic obstructive pulmonary disease in the Saskatchewan health care datafiles
Stat Med
 , 
1995
, vol. 
14
 
24
(pg. 
2627
-
2643
)
44.
Tennis
P
Bombardier
C
Malcolm
E
, et al.  . 
Validity of rheumatoid arthritis diagnoses listed in the Saskatchewan hospital separations database
J Clin Epidemiol
 , 
1993
, vol. 
46
 
7
(pg. 
675
-
683
)
45.
Tu
K
Campbell
NR
Chen
ZL
, et al.  . 
Accuracy of administrative databases in identifying patients with hypertension
Open Med
 , 
2007
, vol. 
1
 
1
(pg. 
e18
-
e26
)
46.
Utter
GH
Cuny
J
Sama
P
, et al.  . 
Detection of postoperative respiratory failure: how predictive is the Agency for Healthcare Research and Quality's Patient Safety Indicator?
J Am Coll Surg
 , 
2010
, vol. 
211
 
3
(pg. 
347
-
354
)
47.
Sadeghi
B
Baron
R
Zrelak
P
, et al.  . 
Cases of iatrogenic pneumothorax can be identified from ICD-9-CM coded data
Am J Med Qual
 , 
2010
, vol. 
25
 
3
(pg. 
218
-
224
)
48.
White
RH
Garcia
M
Sadeghi
B
, et al.  . 
Evaluation of the predictive value of ICD-9-CM coded administrative data for venous thromboembolism in the United States
Thromb Res.
 , 
2010
, vol. 
126
 
1
(pg. 
61
-
67
)
49.
White
RH
Sadeghi
B
Tancredi
DJ
, et al.  . 
How valid is the ICD-9-CM based AHRQ Patient Safety Indicator for postoperative venous thromboembolism?
Med Care
 , 
2009
, vol. 
47
 
12
(pg. 
1237
-
1243
)
50.
Utter
GH
Zrelak
PA
Baron
R
, et al.  . 
Positive predictive value of the AHRQ accidental puncture or laceration patient safety indicator
Ann Surg
 , 
2009
, vol. 
250
 
6
(pg. 
1041
-
1045
)
51.
Romano
PS
Mull
HJ
Rivard
PE
, et al.  . 
Validity of selected AHRQ patient safety indicators based on VA National Surgical Quality Improvement Program data
Health Serv Res.
 , 
2009
, vol. 
44
 
1
(pg. 
182
-
204
)
52.
Yasmeen
S
Romano
PS
Schembri
ME
, et al.  . 
Accuracy of obstetric diagnoses and procedures in hospital discharge data
Am J Obstet Gynecol
 , 
2006
, vol. 
194
 
4
(pg. 
992
-
1001
)
53.
Landis
JR
Koch
GG
The measurement of observer agreement for categorical data
Biometrics
 , 
1977
, vol. 
33
 
1
(pg. 
159
-
174
)