By use of the National Nosocomial Infections Surveillance (NNIS) System's surgical patient surveillance component protocol, the NNIS basic risk index was examined to predict the risk of a surgical site infection (SSI). The NNIS basic SSI risk index is composed of the following criteria: American Society of Anesthesiologists score of 3, 4, or 5; wound class; and duration of surgery. The effect when a laparoscope was used was also determined. Overall, for 34 of the 44 NNIS procedure categories, SSI rates increased significantly (P < .05) with the number of risk factors present. With regard to cholecystectomy and colon surgery, the SSI rate was significantly lower when the procedure was done laparoscopically within each risk index category. With regard to appendectomy and gastric surgery, use of a laparoscope affected SSI rates only when no other risk factors were present. The NNIS basic SSI index is useful for risk adjustment for a wide variety of procedures. For 4 operations, the use of a laparoscope lowered SSI risk, requiring modification of the NNIS basic SSI risk index.
In recent years, increased attention has been given to measuring clinical outcomes as a component of comprehensive quality assurance programs . A significant impediment to developing meaningful hospital-acquired infection rates that can be used for intra- and interhospital comparisons has been the lack of an adequate means of adjusting for case mix. For surgical site infections (SSI), the traditional wound classification system, which stratifies each wound into 1 of 4 categories—clean, clean-contaminated, contaminated, and dirty-infected—has been available since 1964 [2–4]. Limitations of this system of risk stratification are well recognized. One of the major problems is its failure to account for the intrinsic patient risk of developing an SSI. A composite risk index that captures the joint influence of this and other risk factors is required before meaningful comparisons of SSI rates can be made among surgeons, among institutions, or over time.
A simple index was developed during the Study on the Efficacy of Nosocomial Infection Control (SENIC) project . In that study, highly trained data collectors evaluated >338,000 patient records from a probability sample of hospitals in the United States to calculate infection rates. The hospitals' surveillance and prevention/control programs were also evaluated. The SENIC study found that hospitals with lowest nosocomial infection rates had strong surveillance and prevention/control programs. In particular, the SENIC study developed the first risk index to aid in risk adjustment of infections that occur after surgery and showed that the collection, calculation, and dissemination of surgeon-specific SSI rates to surgeons lowered the SSI rates. The SENIC study provided the strongest scientific evidence to date for the efficacy of the surveillance of SSIs. We previously reported an adaptation of this risk index used by 44 National Nosocomial Infections Surveillance (NNIS) system hospitals that collected data from January 1987 through December 1990 under the surgical patient surveillance component protocol, which includes definitions of eligible patients, operations, and hospital-acquired infections . The NNIS basic SSI risk index was a significantly better predictor of SSI risk than was the traditional wound classification system, and it performed well across a broad range of operative procedures. However, the 1990s witnessed changes in health care delivery with regard to surgical procedures. Considerable numbers of procedures are now done on an outpatient basis, and the surgical patients admitted to hospitals tend to have higher intrinsic risk and are often discharged earlier [7–9]. We examine the effect of these changes and the dramatic expansion of the NNIS system on the ability of the NNIS basic SSI risk index to predict the risk of an SSI following an operative procedure.
The surveillance protocols used by hospitals in the NNIS system, including the mapping of International Classification of Diseases, 9th revision, codes into the 44 NNIS operative procedure categories, have previously been described . Under the surgical patient surveillance component, all patients undergoing operations in preselected operative procedure categories are monitored, at least until discharge, for postoperative infections. Standard definitions for hospital-acquired infections are used .
In the NNIS basic SSI risk index, each operation is scored by counting the number of risk factors present among a patient having an American Society of Anesthesiologists (ASA) preoperative assessment score of 3, 4, or 5; an operation classified as either contaminated or dirty-infected; and an operation with duration of >T h, where T depends on the operative procedure being done.
The ASA score assesses preoperatively the overall physical status of the patient and is itself a scoring system. Preoperatively, patients are assessed and given a score that ranges from 1 (for an otherwise healthy patient) to 5 (for a patient not expected to survive the next 24 h) [12–14]. The distribution of duration of operation for the different operative procedures was determined. The 75th percentile of each distribution was identified, rounded to the nearest whole number of hours, and used as the cut point, T, for distinguishing between operations of short and long duration. The NNIS basic SSI risk index has values of 0, 1, 2, or 3. Risk categories were defined by combining adjacent risk index values when no significant difference in SSI risk was found between them (table 1).
To summarize the strength of the association between a potential risk factor, or the composite risk index, and a patient's risk of developing an SSI, we calculated the Goodman-Kruskal (G) statistic . Ranging from -1 to +1, this nonparametric correlation coefficient is most useful for comparing the relative predictive power of different risk factors or comparing a risk factor with the composite index.
From 1992 through 1998, several changes were made in the data collection process of the NNIS system, and these changes had an impact on the nature of the data available for analysis. In January 1992, patients undergoing coronary artery bypass graft procedures were categorized into those with 2 incisions (chest and donor vessel site) and those with only a chest incision (e.g., internal mammary arteries used for grafting). In 1992, we also began to identify operative procedures done via a fiber-optic scope.
In June 1994, we subdivided spinal fusion/laminectomy procedures into either fusion or laminectomy. In addition, orthopedic procedures that involved prosthetic implants, which had previously been combined, were divided into 3 groups: hip, knee, or other prosthetic orthopedic procedures. In June 1994, discharge date was required on all operative records. We also began collecting a data field called “detected” for all SSIs. This field required NNIS hospitals to indicate whether the SSI had been detected during the same hospital admission as the operative procedure, while performing postdischarge surveillance, or on readmission to the hospital. No formal postdischarge surveillance protocol was developed for the NNIS system, because no method has been shown to be optimal and the participating hospitals indicated that the most effective postdischarge surveillance methods are likely to vary according to each institution's setting. The data we used herein were reported by NNIS hospitals, each of which followed the surgical patient surveillance component protocol for ≥1 month from the period from January 1992 through June 1998.
Data regarding 738,398 NNIS operative procedures performed during January 1992 through June 1998, including 19,267 subsequent SSIs, were reported from 225 NNIS hospitals. More than 63% of these procedures were done during the period of 1995–1998.
The NNIS basic SSI risk index: duration of surgery. The cut point for duration of surgery ranged from 1 h, for appendectomy, limb amputation, and cesarean section, to 5 h, for coronary artery bypass graft (chest and donor vessel site) and cardiac surgery, and 7 h, for organ transplantation (table 1).
From 1987–1991 to 1992–1998, the 75th percentiles for the 44 NNIS operative procedure categories changed very little (<20 min), except for head and neck surgery, for which the cut point increased from 5 h to 7 h. The 8 procedures with changes in cut points were procedures for which the cut points had been close to the half hour and, when rounded to the nearest integer for the hour, had a greater chance of changing over time despite the small change in duration.
Utility of the NNIS basic SSI risk index. The NNIS basic SSI index was a useful method for risk adjustment for a wide variety of procedures (table 1). Overall, for 34 of the 44 NNIS operative procedure categories, SSI rates increased significantly (P < .05) as the number of risk factors increased, on the basis of Goodman-Kruskal coefficients (±SEM, 0.33 ± 0.006; P < .0001 for all procedures combined). The NNIS basic SSI risk index was a particularly useful method of risk adjustment for 28 of the 31 NNIS operative procedures that were not part of a category designated as “other,” such as “other genitourinary system procedures.” The 3 procedures that were not part of a category designated as “other” for which there was no significant increase in SSI rates with increasing numbers of risk factors were nephrectomy, splenectomy, and limb amputation.
The NNIS basic SSI index was not as useful for NNIS operative procedures that combined a variety of operations. For 6 of the 13 combinations of procedures designated as “other,” SSI rates increased significantly (P < .05) with increasing numbers of risk factors. These 6 were other cardiovascular system procedures; other genitourinary system procedures; other ear, nose, and throat procedures; other gastrointestinal procedures; other musculoskeletal procedures; and other endocrine procedures. For 4 procedures, analysis suggested the need to incorporate an additional measure into the SSI index: the use of the laparoscope.
Use of the laparoscope. Since 1992, laparoscopes have been increasingly used in surgery. From 1992 through 1997 (the last complete year for data collection), the most common procedures done laparoscopically were cholecystectomy operations, with the laparoscope being used 64% of the time. Laparoscope use for cholecystectomy operations increased steadily from 59% in 1992 to 72% in 1997, and SSI rates were significantly lower when a laparoscope was used (0.6% vs. 1.8%; P < .001).
Other procedures frequently done by means of a laparoscopic (or endoscopic) approach during the period of 1992–1998 were the following: appendectomy (19%), vaginal hysterectomy (15%), gastrectomy (8.5%), exploratory laparotomy (6.9%), herniorrhaphy (4.5%), ventricular-peritoneal shunt (4.2%), and colon surgery (2.6%). NNIS hospitals also reported the use of a laparoscope in other procedures, but in general, the use was <2% of the procedures reported or the NNIS operative procedure was a combination of procedures (such as “other ear, nose, or throat” surgery, with a reported laparoscope use of 14%), and the combination of procedures did not show a significant difference in SSI rates with laparoscope use.
Modified NNIS basic SSI risk index for cholecystectomy, colon surgery, appendectomy, and gastric surgery: the importance of the laparoscope. Because the use of a laparoscope has the potential for lowering the risk of SSI, we investigated this possibility for each of the procedures identified in the previous section as having significantly lower overall rates when a laparoscope was used. For only 4 NNIS operative procedures did we find it necessary to incorporate laparoscope use into the risk index: cholecystectomy, colon surgery, appendectomy, and gastric surgery. For cholecystectomy, within each of the basic SSI risk index categories, the SSI rate was significantly lower when a laparoscope was used (table 2). Moreover, as table 2 suggests, the influence of the laparoscope could be captured by simply subtracting 1 from the basic SSI risk index whenever the procedure was done laparoscopically. With this modification, the risk index has values of M (or -1), 0, 1, 2, or 3; the SSI rate in each category is significantly lower than the rate in the next category (table 3).
Only 2.6% of colon surgery procedures were done laparoscopically. The influence of laparoscope use on SSI was the same as that for cholecystectomy. Hence, the NNIS modified SSI risk index could be defined by subtracting 1 from the basic SSI risk index whenever colon surgery operations were done laparoscopically, once again yielding risk categories with values of M (or -1), 0, 1, 2, or 3. All of the SSI rates were significantly different among the 5 risk categories for colon surgery (table 3).
For appendectomy, laparoscope use was high (19%). SSI rates were not significantly lower when the laparoscope was used within each of the basic SSI risk index categories, except for 0; this was unlike the pattern for cholecystectomy and colon surgery. The use of a laparoscope did not call for subtracting 1 from the basic SSI risk index, except when the index was 0. Hence, we split the risk category 0 group into “0-No” and “0-Yes” and otherwise ignored whether the procedure was done laparoscopically. SSI rates with 2 or 3 of the other risk factors (ASA score, wound class, or duration of surgery) did not differ significantly, and the data were combined to form a single category 2,3 (table 3). Therefore, there were 4 risk categories for appendectomy: 0-Yes, 0-No, 1, and 2,3, where “Yes” or “No” refers to the appendectomy having been done with or without a laparoscope. For gastric surgery, a laparoscope was used ~8.5% of the time. As seen for appendectomy, the use of a laparoscope significantly reduced the SSI risk only when the risk category was 0, thereby yielding 4 risk categories for gastric surgery: 0-Yes, 0-No, 1, and 2,3 (table 3).
Postdischarge surveillance of SSIs. Of the 19,267 SSIs, only 14,949 (78%) had a recorded value in the category “detected,” because this variable was not collected until 1994. Of these 14,949 SSIs, 46% were detected during the current admission, 16% through postdischarge surveillance efforts, and 38% on readmission (figure 1).
The more serious SSIs were detected before discharge or on readmission. Of 2392 SSIs detected by infection control professionals at NNIS hospitals in the postdischarge outpatient setting, 78% were skin infections, 13% were deep incisional infections, and 9% were organ/space infections (figure 2). In contrast, the distribution of 6876 SSIs detected before discharge was 43% skin infections, 19% deep incisional infections, and 38% organ/space infections, and the distribution of 5681 SSIs detected on readmission was 40% skin infections, 31% deep incisional infections, and 29% organ/space infections.
A number of studies have reported a decrease in the incidence of SSIs when surveillance programs have been implemented that included the feedback of postoperative wound infection rates to practicing surgeons [16–19]. Indeed, the SENIC project showed such feedback to be an essential component of an effective infection control program . Warnings have been sounded regarding an overly simplistic approach to the calculation and comparison of surgeon-specific wound infection rates . The results of applying the basic SSI risk index to NNIS data reaffirm the general conclusions drawn from the SENIC risk index and our previous report . When applied to a more recently collected set of data, which was subject to the normal interhospital variations in case-finding methods, diagnostic accuracy, and risk factor misclassification error, the NNIS basic SSI risk index proved useful in risk adjustment for most procedures. Furthermore, this analysis emphasized the need to incorporate risk factors other than the traditional wound classification into a composite index of SSI risk before attempting to compare infection rates among surgeons, among institutions, or across time. The ASA score is a critical component of the index, included in an attempt to measure intrinsic host susceptibility. Somewhat analogous to the number of discharge diagnoses used in the SENIC index, the ASA score has the advantage of being readily available at the time of surgery. The approximate 75th percentile of duration of operation provides the index with additional discriminatory power when applied to specific operative procedures, such as coronary artery bypass grafts. Of interest was the remarkable lack of change in the duration of surgery cut points in this report compared with those in our previous report . Cardo et al.  found that the accuracy of surgical team members in assessing wound classification for general and trauma surgery was 88% (95% CI, 82%–94%). The accuracy of recording the duration of operation (i.e., time from skin incision to skin closure) and the ASA class has not been studied.
Our report also demonstrates the value of including an additional variable in the index, namely the use of the laparoscope. Although only 4 procedures in the index use the laparoscope at this time, we expect that number to increase.
The results in table 1 can be used by hospitals in several ways. For each surgeon, procedure-specific SSI rates can be calculated and compared against the corresponding rates in table 1 by means of a simple Z test or Fisher's exact test . In many hospitals, the number of procedures done by an individual surgeon in some of the risk categories may be small. In that event, one can calculate how many infections would have been “expected” to occur among patients, taking into account the type and number of procedures and the risk categories of the patients. The expected number of SSIs can be obtained by multiplying the number of operations done by the surgeon in each procedure risk category by the NNIS rate for that same procedure risk category and then dividing by 100. By summing the numbers of expected SSIs for the procedure and risk categories in which surgery was done, we can compare it with the number of observed SSIs for the surgeon. The ratio of the observed number of SSIs (O) that occurred to the expected number (E) is called the standardized infection ratio or SIR: SIR = O/E.
The SIR is deceptively simple. It is an easy way to interpret summary measures of the SSI experience of an individual surgeon, service, or hospital. Values that exceed 1.0 indicate that more SSIs occurred than were expected (and by how much), whereas values of <1.0 indicate the opposite. In calculating the expected number of SSIs, we account for the type of procedures performed and the distribution of patients by risk index, that is, the case mix. Therefore, the SIR is a risk-adjusted summary measure and can be used for comparative purposes. To test whether the SIR differs significantly from its nominal value of 1.0, a Z statistic can be calculated by the following formula (valid as long as E ≤ 1): Z = 2(√O - √E).
The SIRs for 2 surgeons or for the same surgeon during 2 time periods can also be compared .
The value of comparative SSI rate data depends largely on the accuracy and consistency with which the data are collected. If SSI rates vary because of differences in postdischarge surveillance intensity, then the value of the comparison is diminished. Our data suggest that SSI rates are not generally correlated with postdischarge surveillance intensity, as measured by the percentage of SSIs detected after discharge among patients who are not readmitted to the hospital. Previous studies have shown that 12%–84% of SSIs are detected after patients are discharged from the hospital [16, 23–41]. Postdischarge surveillance methods have been used with varying degrees of success for different procedures and among hospitals, and they include direct examination of patients' wounds during follow-up visits to either surgery clinics or physicians' offices [8, 18, 23, 16, 30, 37, 42–44], review of medical records of surgery clinic patients [29, 42, 45], administration of questionnaires to patients by mail or telephone [8, 25, 27, 28, 31, 32, 38, 46–48], or administration of questionnaires to surgeons by mail or telephone [8, 16, 24, 27–29, 33, 34, 36, 38, 45]. One study found that patients have difficulty assessing their wounds for infection (specificity, 52%; positive predictive value, 26%) , suggesting that data obtained by use of the patient questionnaire may inaccurately represent SSI rates. A review suggested that there is no consensus for monitoring SSIs after discharge from the hospital .
Recently, Sands et al.  performed a computerized search of 3 databases: ambulatory encounter records for diagnostic, testing, and treatment codes; pharmacy records for specific antimicrobial prescriptions; and administrative records for rehospitalizations and emergency room visits. The purpose of the search was to determine which database best identified SSIs. They found that pharmacy records indicating that a patient had received antimicrobial agents commonly used to treat soft tissue infections had the highest sensitivity and positive predictive value. Their study also showed the low sensitivity of many of the other case-finding methods after discharge. During the period of 1992–1998, most NNIS hospitals did not have access to pharmacy records for information on specific antimicrobial prescriptions after discharge. This suggests that postdischarge case finding by most NNIS hospitals may have been consistently insensitive.
The general applicability of the NNIS basic SSI risk index within a broad range of operative procedure categories, with the inclusion of laparoscope use where needed, is encouraging, but there remains room for improvement. Almost certainly, additional important risk factors for specific procedures need to be identified and incorporated into such an index , such as antibiotic prophylaxis during procedures in which it has been shown to be effective but not universally adopted [52–57].
It is important to keep in mind the limitations of any risk index. A statistically significant difference between a risk index–adjusted SSI rate for a cohort of patients and an appropriate comparison group merely indicates the presence of a potential problem, one generally worthy of further investigation. For a particular group of patients, the index may still not have adequately adjusted for differences in case mix between an individual hospital's group and the comparison group. Also, in the case of the NNIS basic SSI risk index, 2 of the risk factors in the index, wound class and duration of operation, may indirectly reflect quality of care. Adjustment for these factors may mask rather than elucidate a potential problem. A comparison of the distribution of the operations among the risk categories with the distribution in a corresponding group, such as that in table 1, may also be useful.
As we move forward in the development of measures of health care quality, a simple index to predict risk for SSIs will be less optimal for interhospital comparison once more risk factors are ascertained. Interhospital comparisons can be improved by using the SIR and by enhancing the accuracy of the expected number of SSIs from multivariate models by use of aggregated NNIS data. This approach to comparison will allow inclusion of the full range of risk factors for operative procedures. Until then, the NNIS basic SSI risk index remains the best currently available method for benchmark comparisons of SSI rates.