Background. Studies investigating the impact of guideline implementation for inpatient management of community-acquired pneumonia (CAP) usually have methodological limitations. We present a controlled study that compared interventions before and after the implementation of a practice guideline.
Methods. Clinical and demographic characteristics, as well as process-of-care and outcome indicators, were recorded for all patients with CAP who were admitted to Galdakao Hospital (Galdakao, Spain) in the 19-month period after the implementation, on 1 March 2000, of a guideline for the treatment of CAP. These data were also recorded for all patients with CAP who were admitted to this hospital during the year before the guideline was implemented, as well as for randomly selected inpatients with CAP at 4 other hospitals during both periods (i.e., before and after guideline implementation) who were chosen as an external comparison group. Multivariate linear and logistic regression models were employed for adjustment.
Results. Guideline implementation resulted in shorter durations of antibiotic treatment (P < .001) and intravenous treatment (P < .001), better coverage of atypical pathogens (P < .001), and improved appropriateness of antibiotic treatment (P < .001), compared with the period before the guideline was implemented. The adjusted analyses revealed decreases in 30-day mortality (odds ratio [OR], 2.14; 95% confidence interval [CI], 1.23–3.72) and in-hospital mortality (OR, 2.46; 95% CI, 1.37–4.41) and a 1.8-day reduction in the duration of hospital stay. In the control hospitals, there were small but statistically insignificant changes in these indicators for admitted patients.
Conclusions. This study, which was performed with an adequate, controlled before-and-after intervention design, demonstrated significant improvements in both process-of-care and outcome indicators after implementation of a guideline for treating CAP.
The aim of best-practice guidelines is to improve the efficiency and effectiveness of health care by reducing variation in key aspects of care . However, guidelines are only effective if there is compelling evidence that links health-care processes to outcomes. This is the case with community-acquired pneumonia (CAP) that requires hospitalization [2–5].
Several studies have shown that implementing a guideline for the management of patients hospitalized for the treatment of CAP significantly improves the process of care [6–16]. However, some studies overestimated the impact of guideline implementation, sometimes by failing to control for secular trends [6–12]. Other studies had limited scope, such as analysis of the hospital admission decision  or the indicators related to the process of care . The most methodologically robust study we are aware of examined the use of resources as an outcome .
The goal of our study was to examine the full impact of the implementation of a practice guideline for the treatment of CAP. For this purpose, we employed a before-and-after intervention design, as well as an external control group, to determine if guideline implementation improved both the process-of-care and the final outcome above and beyond any concurrent secular trends.
Patients and Methods
Study sites. This study was performed in 5 different hospitals, all of which belong to the network of public hospitals of the Basque Health Care Service. These hospitals provide free, unrestricted care to nearly 100% of the population in their catchment areas. The intervention center was Galdakao Hospital, a 400-bed teaching hospital in the province of Bizkaia, Basque Country (northern Spain), which serves a population of 300,000 inhabitants. The 4 centers from which patients who served as the external control group were enrolled are also university hospitals and thus share technological and human resources that are similar to those of the intervention center. These 4 hospitals serve a total population of 1,800,000 people.
Study design. The study was designed as a controlled before-and-after intervention study. The intervention cohort was composed of all patients with CAP who were admitted to Galdakao Hospital during the 19-month period after the implementation of a practice guideline on 1 March 2000. This interval is defined as the follow-up period. A preintervention comparison group (i.e., the preintervention cohort) included all patients with CAP who were admitted to Galdakao Hospital between 1 March 1998 and 1 March 1999. This interval is defined as the baseline period. No patients who were admitted during the year immediately preceding the guideline implementation were included in this control group, to eliminate the possible influence of the development of the practice guideline.
To provide an external control, we selected a random sample of patients with CAP who were admitted to 4 similar hospitals during the same 2 periods—control cohort 1 was comprised of patients who were admitted during the year-long baseline period, and control cohort 2 was comprised of patients who were admitted during the 19-month follow-up period. Sample size estimates were made on the basis of previous hypotheses about expected differences in main process-of-care and outcome results, with α = 0.05 and 1 - β = 0.8. A total of 250 patients were selected from each center: 100 were admitted during the baseline period, and 150 were admitted during the follow-up period. Patients from 1 of the 4 comparison hospitals were enrolled by means of a different sampling method (140 patients were from the baseline period, and 200 were from the follow-up period) because of its larger size. To take into account expected losses (due to case confirmation and exclusion criteria) and missing records, we oversampled by 40% the sample size estimated from our previous hypothesis.
Patients. All adult patients (age, ⩾18 years) with CAP who were admitted to one of the hospitals were included in the study if CAP was suspected within the first 24 h after arrival in the emergency department. Pneumonia was defined by the presence of pulmonary infiltrates on a chest radiograph that were not known to be old and symptoms that were consistent with pneumonia, including cough, dyspnea, fever, and/or pleuritic chest pain. Patients with pneumonia were excluded if they were known to have tested positive for HIV, were chronically immunosuppressed (i.e., if they had undergone solid organ transplantation or splenectomy, had received 10 mg/day of prednisone or the equivalent for >30 days, had received treatment with other immunosuppressive agents, or were neutropenic [neutrophil count, <1.0 × 109 neutrophils/L]), or had been hospitalized during the previous 14 days.
Retrospective selection. We reviewed retrospectively the clinical history of patients with CAP who were admitted to Galdakao Hospital during the baseline period and patients who were randomly selected from the 4 control hospitals during both study periods. Patients with potential pneumonia cases were identified if they had a principal discharge diagnosis of pneumonia (International Classification of Disease, Ninth Revision, Clinical Modification [ICD-9-CM] codes 480.0–480.9, 481, 482.0–482.9, 483.0–483.8, 485, 486, 487.0, and 507.0) or a principal discharge diagnosis of respiratory failure (ICD-9-CM code 518.81) and a secondary diagnosis of pneumonia. Case confirmation required that the patient have an appropriate ICD-9-CM code; suspected CAP, based on results of diagnostic tests, during the first 24 h after arrival in the emergency department; clinician-documented diagnosis of pneumonia; and chest radiograph findings consistent with CAP.
Each chart was reviewed and data were abstracted by 2 trained reviewers using a structured questionnaire form. A member of the research team (P.P.E.) reviewed a sample of 40 records for all variables and checked the most relevant ones to test the accuracy of the data retrieved by the reviewers. Also, the 2 reviewers examined the same variables in 40 additional records to test for agreement between the reviewers on the most relevant variables. We found κ and intraclass correlation coefficients to be ⩾0.98 in all cases.
To provide additional quality control, we reviewed retrospectively the clinical history of patients with CAP who were admitted to Galdakao Hospital during the intervention period. The additional patients who were identified in this review and who fulfilled the study criteria (22 patients) were included in the study.
Study intervention. The 4 key points of the intervention that followed from the institution of a practice guideline for CAP were (1) the establishment of explicit admission-decision criteria on the basis of risk classes specified by the Pneumonia Severity Index (PSI)  and by a series of additional criteria  (to reduce the percentage of admissions of patients with less-severe CAP [PSI classes I, II, and III]); (2) the use of early, appropriate antibiotic treatment based on guidelines established by the Spanish Society of Pneumology and Thoracic Surgery , which are similar to those of the American Thoracic Society ; (3) the timing of the switch from intravenous to oral antibiotics ; and (4) the criteria for hospital discharge .
Process-of-care and outcome measurements. The clinical and demographic characteristics of all patients were recorded, as were all variables included in the PSI score  and previous antibiotic treatment. For calculation of the PSI score, missing data and hypothesized results of unperformed laboratory tests were considered to within normal ranges.
Process-of-care variables included (1) initial antibiotic treatment that was consistent with the recommendations of American Thoracic Society or the Infectious Diseases Society of America [20, 21], (2) coverage of atypical pathogens (including treatment with macrolides or levofloxacin and similar agents), (3) antibiotic administration within 8 h after arrival in the emergency department, (4) total duration of antibiotic therapy, and (5) duration of intravenous antibiotic therapy.
Outcome measurements included (1) vital status immediately after and 30 days after discharge, (2) admission to the intensive care unit, (3) use of mechanical ventilation, (4) septic shock (defined as systolic arterial tension <90 mm Hg and use of vasopressors for a minimum of 4 h), (5) hospital readmission because of pneumonia-related complications within 30 days after discharge (medical records of all readmitted patients were independently evaluated by 2 trained pneumologists), and (6) duration of stay (calculated as the discharge date minus the admission date). Vital status was evaluated by use of medical records and a regional administrative database.
Statistical methods. Descriptive statistics included frequencies, percentages, means, medians, and standard deviations. To compare categorical variables in the intervention and control hospital cohorts during the 2 study periods, Fisher's exact test and χ2 analysis were employed. To analyze continuous variables during the study periods, Student's t test and a nonparametric test (the Wilcoxon rank sum test) were employed.
To compare process-of-care and outcome parameters in the intervention and control hospitals during the 2 study periods, we also conducted adjusted analyses. We also compared patients from the preintervention hospital with patients from cohort 1 by means of adjusted analysis. To analyze the continuous dependent variables (durations of antibiotic therapy, intravenous therapy, and hospitalization), multivariate linear regression models were employed. Because of the nonnormal distribution of these dependent variables, log transformation was performed. Parameter estimates are presented after exponentiation. For analysis of all other dependent variables, dichotomous, multivariate logistic regression models were employed. ORs and 95% CIs are presented. All models were adjusted for severity of illness (which was measured by use of the PSI score as a continuous variable), multilobar radiological involvement, antecedents of chronic obstructive pulmonary disease, and antibiotic treatment received before hospital admission.
For the main comparisons of process-of-care and outcome parameters, 4 groups were established: 1 group of patients who were treated according to the guideline (the intervention cohort) and 3 groups of patients who were treated according to standing orders (the preintervention cohort at the intervention hospital and cohorts 1 and 2 at the control hospitals). In all models, the comparison group was the intervention cohort. All models were adjusted for severity of illness.
The intervention cohort consisted of 417 patients with diagnoses of pneumonia who were admitted to the intervention hospital. The preintervention cohort included 377 (75.1%) of a total of 502 patients who received a diagnosis of pneumonia at the time of hospital discharge (90.2% of the cases were confirmed, with an exclusion rate of 15.1%).
Of the 2366 patients in the control hospitals who received a discharge diagnosis of pneumonia during the baseline period, 614 (26%) were randomly selected, and, of these, 467 patients (76.1%) (control cohort 1) were included in the study (90.9% of the cases were confirmed, with an exclusion rate of 16.3%). Of the 3811 patients who received a discharge diagnosis of pneumonia during the intervention period, 861 (22.6%) were randomly selected, and 654 (76%) (control cohort 2) were included in the study (89.8% of the cases were confirmed, with an exclusion rate of 15.4%). Differences in case confirmation and exclusion rates between the 3 retrospectively selected groups were not statistically significant.
Patient characteristics. The characteristics of patients in control cohorts 1 and 2 were similar to each other and to those of patients in the preintervention cohort (table 1). Among all patients who were admitted to the intervention hospital, those in the intervention cohort were significantly older and had higher PSI risk scores than patients in the preintervention cohort (mean PSI, 98.9 vs. 87.3); fewer patients in the former group belonged to risk classes I and II. Differences in PSI scores were also found between patients in the intervention cohort and those in control cohort 2 (mean PSI risk score, 98.9 vs. 93.1; P = .01).
Performance of the process-of-care indicators. The performance of process-of-care indicators are shown for both study periods in table 2. For the control hospitals, there were no significant changes in the mean indicators between control cohorts 1 and 2. For the intervention hospital, there was a statistically significant improvement in all but a single medical indicator after guideline implementation. Reductions were observed in the intervention cohort with respect to the duration of antibiotic treatment (11.4 vs. 12.9 days) and the duration of intravenous treatment (3.2 vs. 4.5 days). Among patients with the most serious cases of disease (PSI classes IV and V), the reductions in the duration of antibiotic treatment and intravenous treatment in the intervention cohort each averaged >2 days. No difference was observed between the preintervention and intervention cohorts with respect to receipt of antibiotic administration within 8 h after presentation to the hospital.
Outcome indicators. Indicators on the outcomes of care are shown for both study periods in table 3. For the control hospital cohorts, no changes were observed in the indicators recorded for admitted patients in both adjusted and unadjusted analyses, except that a higher percentage of patients with septic shock was found in control cohort 2. No significant differences in 30-day mortality or in-hospital mortality were seen between control cohorts 1 and 2.
Before guideline implementation, mortality did not differ between patients in the intervention hospital and patients in the control hospitals, as revealed by adjusted and unadjusted analyses. After guideline implementation, adjusted analysis showed a lower 30-day mortality (OR, 2.14; 95% CI, 1.23–3.72; P < .01) and in-hospital mortality (OR, 2.46; 95% CI, 1.37–4.41; P < .01) in the intervention cohort, compared with the preintervention cohort.
The duration of hospitalization differed between the preintervention cohort and control cohort 1 (adjusted mean duration of stay, 6.4 vs. 7.8 days; P < .001). Unadjusted and adjusted analyses revealed statistically significant decreases in the mean duration of stay after guideline implementation in the intervention cohort, compared with the preintervention cohort (adjusted mean duration of stay, 4.8 vs. 6.7 days; P < .001). This was also the case among the most seriously ill patients (i.e., those with a PSI risk of IV or V) (adjusted mean duration of stay, 5.4 vs. 7.4 days; P < .001).
Adjusted and unadjusted analyses revealed no statistically significant differences between cohorts at the intervention hospital with respect to important clinical results, such as admission to the intensive care unit, use of mechanical ventilation, septic shock, or 30-day readmission.
Effects of guideline implementation. Table 4 summarizes a comparison of patients who were treated according to the guideline (the intervention cohort) with patients who were treated according to standing orders (the preintervention cohort at the intervention hospital and cohorts 1 and 2 at the control hospitals). Adjusted analyses revealed that 30-day mortality and in-hospital mortality were lower among patients in the intervention cohort, compared with patients in the preintervention cohorts, control cohort 1, and control cohort 2. The differences were statistically significant in comparisons between the intervention cohort and the preintervention cohort and control cohort 2.
Patients in the intervention group also experienced reductions in the duration of stay (adjusted mean duration of stay, 4.7 vs. 6.5, 8, and 7.6 days, in the preintervention cohort, control cohort 1, and control cohort 2, respectively; P < .001). Other significant differences were observed for patients in the intervention cohort with respect to the duration of intravenous treatment (adjusted mean duration of stay, 2.6 vs. 3.9, 5.2, and 5.3 days; P < .001), the duration of antibiotic treatment (adjusted mean duration of stay, 11.2 vs. 12.5, 14.2, and 14.1 days; P < .001), and the increase in the coverage of atypical pathogens (P < .001).
The implementation of a guideline for the treatment of CAP had a positive impact on the treatment of patients admitted to the hospital with this condition. The strength of this study is its pre- and postintervention design that also included an external control group. Multiple comparison groups allowed us to overcome a design problem of previous studies and enabled us to demonstrate that the improvements were due to the guideline implementation rather than to secular trend.
The implementation of a practice guideline for in-patient treatment of CAP was associated with reductions in mortality, average duration of hospitalization, and duration of hospitalization for patients with the most serious CAP (PSI classes IV and V), even after adjusting for severity of illness. Guideline implementation also appeared to improve process-of-care factors in patients hospitalized with CAP. Our results are consistent with those of a number of studies that have supported the positive effect of guidelines on the performance of processes of care and outcome for patients hospitalized with CAP [6–16].
We observed lower rates of 30-day and in-hospital mortality among inpatients with CAP after guideline implementation, which confirms the results of previous studies [6, 22]. Explanations for this decreased mortality may include the identification of patients with severe pneumonia at admission to the hospital, the selection of appropriate antibiotics, and/or the coverage of atypical pathogens [2, 4, 23, 24]. The design of this study did not allow us to test these hypotheses.
The reduction in the average duration of hospitalization confirms the results obtained in previous uncontrolled observational studies [6–12] and are similar to those from a randomized trial conducted in 19 Canadian hospitals . The duration of hospital stay for patient with CAP depends on the time needed to reach clinical stability, which is significantly influenced by the severity of the disease . Patients with the most severe pneumonia take the longest to recover. However, our results, like those of Castro-Guardiola et al. , indicate that a number of patients reach clinical stability quite quickly and thus can be switched to oral antibiotic therapy and discharged within 24 h after admission. This highlights the opportunities that may exist for improving the management of severe pneumonia among inpatients.
The implementation of the practice guideline caused significant changes in the management of antibiotics and also identified areas for improvement. The number of days for which intravenous antibiotics was required was reduced; the mean duration of intravenous therapy in our study (3.2 days) was consistent with that reported elsewhere [16, 25, 27]. We also observed a reduction in the overall duration of antibiotic treatment. Although few data exist with regard to this issue, our results correspond to recommendations found in the latest practice guideline [28, 29].
The rate of antibiotic administration within 8 h after hospital arrival was low for all hospitals, especially the intervention hospital, and did not improve significantly after implementation of the practice guideline. These findings underscore important opportunities for improving the care of patients.
Our study has some limitations. First, medical record documentation and chart review could have introduced flaws. However, our interabstractor reliability was measured and found to be excellent. The outcome indicators we used are highly objective end points that were available for all patients. Similarly, the identification of cases of CAP was probably unbiased, because case confirmation required clinician-documented diagnosis and chest radiograph findings that were consistent with the diagnosis. Second, we did not include patients with a primary diagnosis of sepsis and a secondary diagnosis of pneumonia. However, the number of patients was very small (<0.5% of all patients), and thus we consider that any bias their exclusion may have engendered in the original study was minimal. Third, although we included important outcome measurements, we did not measure other outcomes that may also be important, such as the resolution of symptoms, return to work, or baseline level of activity. Fourth, although we used a validated pneumonia-specific index to adjust for severity of illness, we may not have adjusted fully for differences between patients. Fifth, the study design allowed us to gather data about patients with CAP in the intervention cohort who were discharged from the emergency department and were treated as outpatients, but such data were not available from the retrospective review. Thus, we were unable to estimate the overall impact of the guideline in the reduction of hospital admissions. Nevertheless, after the guideline was implemented a lower percentage of patients with low-risk PSIs (classes I and II) were admitted for treatment of CAP, which suggests a reduction in the overall number of admissions. This is consistent with the findings of other studies [12–14]. Sixth, the intervention was performed in only a single hospital, which may make our results less generalizable. However, the clinical characteristics of the intervention cohort did not differ from some that were used in studies in the United States  and Europe .
In conclusion, the implications of our work are that proper implementation of a practice guideline improved the treatment of patients hospitalized for CAP; reduced mortality rates and duration of hospital stay, even for those patients with the most severe disease; and also allowed us to positively affect key indicators of the process of care, thus guaranteeing a better clinical practice. Until the necessary randomized trials are conducted that can confirm these results, our data suggest that the implementation of and adherence to a practice guideline can significantly improve CAP treatment.
We thank the Direction of Osakidetza, the staff members of the different services, and the medical record sections of the participating hospitals; Inmaculada Arostegui (Departamento de Matemática Aplicada, Estadística e Investigación Operativa, Universidad del País Vasco, Lejona, Spain), for review of the statistical analyses; and Patrick J. Skerrett, for his assistance in editing the manuscript.
Conflict of interest. All authors: No conflict.