Abstract

Objective Develop and evaluate an automated case detection and response triggering system to monitor patients every 5 min and identify early signs of physiologic deterioration.

Materials and methods A 2-year prospective, observational study at a large level 1 trauma center. All patients admitted to a 33-bed medical and oncology floor (A) and a 33-bed non-intensive care unit (ICU) surgical trauma floor (B) were monitored. During the intervention year, pager alerts of early physiologic deterioration were automatically sent to charge nurses along with access to a graphical point-of-care web page to facilitate patient evaluation.

Results Nurses reported the positive predictive value of alerts was 91–100% depending on erroneous data presence. Unit A patients were significantly older and had significantly more comorbidities than unit B patients. During the intervention year, unit A patients had a significant increase in length of stay, more transfers to ICU (p = 0.23), and significantly more medical emergency team (MET) calls (p = 0.0008), and significantly fewer died (p = 0.044) compared to the pre-intervention year. No significant differences were found on unit B.

Conclusions We monitored patients every 5 min and provided automated pages of early physiologic deterioration. This before–after study found a significant increase in MET calls and a significant decrease in mortality only in the unit with older patients with multiple comorbidities, and thus further study is warranted to detect potential confounding. Moreover, nurses reported the graphical alerts provided information needed to quickly evaluate patients, and they felt more confident about their assessment and more comfortable requesting help.

Introduction

As the average age of the US population increases, so does the complexity of their medical care. As a result, up to 5% of patients experience physiologic deterioration during their hospital stay resulting in admission to the intensive care unit (ICU) or death.1,2 These adverse events frequently occur due to a mismatch between the patient's condition and resources available in their current hospital location.3 Studies reveal many adverse events are preceded by indicators of physiologic deterioration including tachypnea, tachycardia, hypotension, decreased oxygen saturation, and changes in level of consciousness.4–12

To pre-empt such adverse events, rapid response systems (RRS), defined as rapid response teams (RRTs) or medical emergency teams (METs, which include physicians), were developed to prevent patient crisis before a cardiopulmonary arrest.13 Critical care outreach teams reported in the literature function in a manner similar to RRS while also providing some outreach to identify high-risk patients.14 RRS calls are usually initiated by bedside nurses caring for the patient.15,16

To be effective, RRS must have an afferent limb (case detection and response triggering) in addition to an efferent limb (medical response) and must be constantly available.3 Numerous studies describe a variety of characteristics for RRS,17–24 and some report various abilities to prevent adverse events1,25–33 while some find no effect.19,34–39 The controversy regarding the benefits of an RRS centers around the recognition of physiologic deterioration as soon as possible and not just on the medical response.8,17,40,41 Thus, a significant barrier to prevent in-hospital cardiopulmonary arrest is the inability to provide adequate monitoring to detect physiologic deterioration.3

To help identify physiologic deterioration early enough to prevent cardiopulmonary arrest, predictive models based on clinical and physiologic data such as the Early Warning Score42 and the Modified Early Warning Score (MEWS)43 have been used. These predictive models assign values to patient data which increase as the values deviate further from normality. Initially, medical staff had to manually collect data and calculate the scores. Due to inconsistencies associated with manual scoring, automated methods, some utilizing response triggering, were developed to facilitate early recognition of physiologic deterioration.44–50 This study reports the results of a 4-year effort to use our electronic medical record (EMR) to develop, implement, and evaluate an automated case detection and response triggering system for physiologic deterioration that meets nursing workflow and endorsement.

Methods

Background

Intermountain Medical Center (IMC) in Salt Lake City, Utah, is an Intermountain Healthcare (IH) level 1 trauma facility and a 472-bed teaching hospital affiliated with the University of Utah School of Medicine. The key feature of the hospital information system is the integrated EMR that contains most clinical information including bedside charting by nurses and interfaces with patient bedside and portable monitors.51–53 The coded data in the EMR facilitates the development and use of clinical decision support programs to analyze the data and constantly monitor patient care.

IMC created a MET when it opened in 2007. Nurses generally call the MET based on subjective assessment of the patient at the bedside as well as integration and recognition of abnormal objective data such as vital signs. Uncertainty inherent in the patient condition and MET process often led to numerous calls, with associated delays, to individuals such as charge nurses, respiratory therapists, or physicians before actual activation of the MET.

Stimulated by information from the MET and non-intensive care nurses’ concerns about when to call the MET, in 2010 we created a MET Risk committee comprised of critical care nurses from the MET and its nursing and medical directors, intensive care physicians, an infectious disease physician, and medical informaticists to investigate the possibility of developing a computer application to monitor hospitalized patients every 5 min and use the latest patient information in our EMR to identify patients with early physiologic deterioration.

Predictive model development

After investigating a number of potential predictive models, the MET Risk committee decided to run multiple models in parallel on real patients and compare their predictive ability. When the initial physiologic deterioration detection models were developed, they were tested and 418 alerts for patients with physiologic deterioration were sent via secure email to the MET director, the MET nurse lead, and two intensive care nurses over an 18-month period (figure 1). The information contained in the email and format was designed by members of the MET Risk committee to help facilitate and simplify the evaluation of the patient. The alert was generated when a patient's score reached a certain level (4 or 5 during development) within a 2 h monitoring window. Hospice patients were excluded from generating alerts. Along with the alert and physiologic total score, the email contained a temporal review of the data for the last 24 h including the overall physiologic score and graphs of the heart rate, systolic blood pressure, temperature, and respiratory rate. Also included were a list of parameters currently out of range, a list of current medications the patient was receiving, and an attached log of all vitals sign measurements and mental status documentation for the previous 4 days (not shown in figure 1).

Figure 1:

Example of the upper part of the information contained in the physiologic deterioration alerts.

Figure 1:

Example of the upper part of the information contained in the physiologic deterioration alerts.

A total of 50 unique patient parameters were included in the different predictive models tested. While the email alerts were sent in real time, the MET and intensive care nurses tried to evaluate each within 72 h. During the model evaluation, the MET Risk committee usually met bi-monthly and reviewed the alerts and compared the models, the logic in each, and the cut off scores for generating alerts. While MEWS seemed to be superior for detecting cases of true physiologic deterioration, we did not feel the positive predictive value was high enough to interrupt bedside nurse's workflow. We then concentrated on modifying MEWS until the positive predictive value would be accepted by nursing staff. The positive predictive value for this study was defined as: did the bedside nurse or MET committee nurse determine the patient to have physiologic deterioration that required additional attention as determined by physiologic or neurologic markers, for example, tachycardia, tachypnea, fever, hypotension, hypertension, or consciousness state.

System development

The physiologic deterioration detection application was initially developed as a Microsoft Windows 64-bit application using the Delphi RAD XE2 (Embarcadero Technologies, San Francisco, CA) development environment. The application polls our EMR every 5 min for data from each patient. Patient data flows into our EMR episodically as clinicians manually chart, from laboratory test results, and every 15–60 min from physiologic monitors attached to patients. The application extracts the pertinent data from the EMR using a client/server system of components that were developed in-house and provided rapid development of interpretive algorithms. When the application identified patients with physiologic deterioration, the email alert described above was immediately sent to certain members of the MET Risk committee for analysis and verification.

Study design

Once we felt the positive predictive value of our modified MEWS physiologic deterioration alerts was acceptable, we met with nursing supervisors and decided to go live on nursing unit A on July 2, 2012 and then on another nursing unit B on September 4, 2012 (figure 2). Nursing unit A is a 33-bed medical and oncology floor and nursing unit B is a 33-bed non-ICU surgical trauma floor. Since the main issue for an RRS to be successful is early case detection and notification of appropriate staff, we determined pager alerts for initial deterioration notification would be more effective than email, supported by the fact that the charge nurses on each nursing unit always carry a pager. After receiving pages, charge nurses went to the patient room, verified the patient, and pulled up a web page on the bedside computer that contained the same information in the same format as sent in the email alerts (figure 1) with all the graphical data from the vital signs, nurse charting of patient mental function, and current medications. The charge nurse discussed the situation with that patient's nurse and made a decision whether or not to notify a physician, a licensed independent practitioner (LIP; physician assistant or nurse practitioner), or call the MET. The nurses caring for patients who were the subject of an alert were asked to fill out an evaluation form and return it to the nurse educator on that unit from the time the study started on that unit until the end of April 2013. The form provided information needed to evaluate the alerts such as which nurse responded to the alert, was the alert correct in identifying physiologic deterioration, was the alert incorrect due to erroneous patient data charted, and if a physician was called. For the cases where a nurse evaluation was not filled out and returned, members of the MET Risk committee retrospectively validated the alert based on the patient's latest medical data at the time of the alert.

Figure 2:

Timeline of the start and end of the physiologic deterioration study including alert intervention, alert evaluation, and simulation training on units A and B.

Figure 2:

Timeline of the start and end of the physiologic deterioration study including alert intervention, alert evaluation, and simulation training on units A and B.

Simulation training

During November 29–30, 2012 and December 3–6, 2012, nursing staff from unit A received additional training on recognizing and caring for deteriorating patients. The 1-hour training sessions covered the history of the MET at IMC, the purpose of the MET, and the patient conditions that necessitate activation of a MET call. Current MET usage data for unit A were shared along with the condition of patients for whom the team was previously activated. Following the instructional session, immersive high-fidelity simulation occurred. Groups were divided with four nurses and two patient care technicians participating in each session. Each group experienced two scenarios: one where they were active participants and one when they observed. Structured debriefing followed each scenario.

Statistical analysis

Two-tailed χ2 tests with Yates’ correction were used to compare differences in proportions such as patient transfers to the ICU, MET calls, and mortality. Two-tailed t tests were used to compare the differences in averages between patient outcomes. To counteract the problem of multiple comparisons in the two different units, the Bonferroni correction was also applied when the uncorrected p value was below 0.05. This study was approved by the Intermountain Healthcare Office of Research for publication.

Results

Physiologic deterioration detection model

The physiologic deterioration model we now clinically use analyzes 11 patient parameters (table 1). Scores ≥4 generate pages sent to the charge nurse. Since it may take time to resolve the physiologic deterioration, the logic in the model only sends one alert for the same patient during a 4 h period. The final model contains five parameters from the patient's vital signs and six different mental status scores charted by nursing staff. Because one of our goals was to achieve nursing approval, the final model for this program was developed based on clinical experience gleaned by prospective examination of the false positive and true positive alerts over an 18-month period rather than retrospective analysis using statistical methods with a defined set of outcome variables. We found we were able to reduce the false positive alerts by altering the weights and values of the variables used by MEWS without the need to increase model complexity with new patient variables available in our EMR.

Table 1:

Final physiologic deterioration model used by Intermountain Healthcare (IH)

Score 1 Point 2 Points 3 Points 4 Points 
SBP, mm Hg 82–90 71–81 or >200 <70  
HR per minute 41–51 or 115–124 125–129 35–40 or 130–200 <35 or >200 
Temperature >38.0°C and <39.0°C 39.0°C   
RR per minute 24–25 26–29 8–9 or (30–66) <8 
O2 O2 increase by 3 L per minute Change from nasal cannula to mask   
Rancho scale Level 3: localized response Level 2: generalized response  Level 1: no response 
Nurse LOC documentation  Arouses with difficulty Requires painful stimuli to arouse Unresponsive/unable to arouse 
GCS Inappropriate Incomprehensible Not verbal  
CAM  Positive   
RASS    −5 no response to verbal or to physical stimulus 
IH sedation score  Moderate sedation Deep sedation Unconscious 
Score 1 Point 2 Points 3 Points 4 Points 
SBP, mm Hg 82–90 71–81 or >200 <70  
HR per minute 41–51 or 115–124 125–129 35–40 or 130–200 <35 or >200 
Temperature >38.0°C and <39.0°C 39.0°C   
RR per minute 24–25 26–29 8–9 or (30–66) <8 
O2 O2 increase by 3 L per minute Change from nasal cannula to mask   
Rancho scale Level 3: localized response Level 2: generalized response  Level 1: no response 
Nurse LOC documentation  Arouses with difficulty Requires painful stimuli to arouse Unresponsive/unable to arouse 
GCS Inappropriate Incomprehensible Not verbal  
CAM  Positive   
RASS    −5 no response to verbal or to physical stimulus 
IH sedation score  Moderate sedation Deep sedation Unconscious 

Alerts are generated when the total score reaches 4 or higher.

CAM, Confusion Assessment Method; GCS, Glasgow Coma Score; HR, heart rate; LOC, level of consciousness; RASS, Richmond Agitation Sedation Scale; RR, respiratory rate; Rancho scale, a cognitive functioning score; SBP, systolic blood pressure.

Nurse evaluation results

A total of 341 (72%) evaluations were returned by nurses on unit A from July 2, 2012 to April 30, 2013 and 127 (58%) from unit B from September 4, 2012 to April 30, 2013 (figure 2). The nurses reported the positive predictive value of the alerts to be 91% including erroneous data in the EMR and 97% for the 319 alerts without erroneous data on unit A and 91% with the erroneous data and 100% for the 115 alerts without erroneous data on unit B. Erroneous data were defined as data entered into the EMR in error and used by the physiologic deterioration score. For the alerts that did not have nursing evaluations returned, the MET committee nurses determined the positive predictive value to be 92% for 133 alerts on unit A and 90% for 91 alerts on unit B.

Nurses also reported they called a physician in unit A 164 times (51%) and 51 times (44%) in unit B for the alerts without erroneous data. Interventions were initiated as a result of an alert in unit A 189 times (59%) and 60 times (52%) in unit B. Diagnostic interventions were initiated for 91 (48%) alerts on unit A, 159 (84%) interventions were therapeutic, and 61 (32%) resulted in both diagnostic and therapeutic interventions. In unit B, 60 interventions were reported by the nurses as a result of the alerts which included diagnostic interventions for 22 (37%) alerts, 48 (80%) therapeutic interventions, and both diagnostic and therapeutic interventions for 10 (17%) alerts.

Physiologic deterioration alerts

During the intervention period from July 2, 2012 to July 1, 2013, a total of 3189 patients were cared for on nursing unit A, while 3100 patients on nursing unit B were cared for from September 4, 2012 to September 3, 2013. A total of 551 physiologic deterioration alerts were generated on nursing unit A and 289 alerts on unit B. Of those, 97 patients generated multiple alerts during their stay on unit A (range of 2–29) and 26 patients on unit B (range of 2–36). Almost 64% of the physiologic deterioration alerts generated had a total score of 4 (table 2). For both nursing units, vital sign data most often contributed to the physiologic deterioration alerts followed by nurse documentation of level of consciousness, Glasgow Coma Score, change in oxygen use, and the Intermountain Healthcare (IH) sedation score (table 3).

Table 2:

Range of physiologic deterioration alert scores for unit A and unit B during the intervention years

Physiologic deterioration total score Total, n (%) Unit A, n (%) Unit B, n (%) 
536 (63.8) 354 (64.2) 182 (63.0) 
166 (19.8) 118 (21.4) 48 (16.6) 
74 (8.8) 42 (7.6) 32 (11.1) 
39 (4.6) 26 (4.7) 13 (4.5) 
13 (1.6) 8 (1.5) 5 (1.7) 
5 (0.6) 1 (0.2) 4 (1.4) 
>10 7 (0.8) 2 (0.4) 5 (1.7) 
Total 840 (100) 551 (100) 289 (100) 
Physiologic deterioration total score Total, n (%) Unit A, n (%) Unit B, n (%) 
536 (63.8) 354 (64.2) 182 (63.0) 
166 (19.8) 118 (21.4) 48 (16.6) 
74 (8.8) 42 (7.6) 32 (11.1) 
39 (4.6) 26 (4.7) 13 (4.5) 
13 (1.6) 8 (1.5) 5 (1.7) 
5 (0.6) 1 (0.2) 4 (1.4) 
>10 7 (0.8) 2 (0.4) 5 (1.7) 
Total 840 (100) 551 (100) 289 (100) 
Table 3:

Number of times each physiologic deterioration model variable contributed to the total score of 4 or higher in unit A and unit B during the intervention years

Variable Total, n (%) Unit A, n (%) Unit B, n (%) 
HR per minute 574 (33.4) 348 (31.5) 226 (36.8) 
RR per minute 517 (30.0) 363 (32.8) 154 (25.0) 
Temperature 366 (21.2) 251 (22.7) 115 (18.7) 
SBP, mm Hg 105 (6.1) 79 (7.1) 26 (4.2) 
Nurse LOC documentation 62 (3.6) 30 (2.7) 32 (5.2) 
GCS 38 (2.2) 12 (1.1) 26 (4.2) 
O2 32 (1.9) 14 (1.3) 18 (2.9) 
IH sedation score 24 (1.4) 8 (0.7) 16 (2.6) 
RASS 2 (0.1) 1 (0.1) 1 (0.2) 
Rancho scale 1 (0.1) 0 (0.0) 1 (0.2) 
CAM 0 (0.0) 0 (0.0) 0 (0.0) 
Total 1721 (100) 1106 (100) 615 (100) 
Variable Total, n (%) Unit A, n (%) Unit B, n (%) 
HR per minute 574 (33.4) 348 (31.5) 226 (36.8) 
RR per minute 517 (30.0) 363 (32.8) 154 (25.0) 
Temperature 366 (21.2) 251 (22.7) 115 (18.7) 
SBP, mm Hg 105 (6.1) 79 (7.1) 26 (4.2) 
Nurse LOC documentation 62 (3.6) 30 (2.7) 32 (5.2) 
GCS 38 (2.2) 12 (1.1) 26 (4.2) 
O2 32 (1.9) 14 (1.3) 18 (2.9) 
IH sedation score 24 (1.4) 8 (0.7) 16 (2.6) 
RASS 2 (0.1) 1 (0.1) 1 (0.2) 
Rancho scale 1 (0.1) 0 (0.0) 1 (0.2) 
CAM 0 (0.0) 0 (0.0) 0 (0.0) 
Total 1721 (100) 1106 (100) 615 (100) 

CAM, Confusion Assessment Method; GCS, Glasgow Coma Score; HR, heart rate; IH, Intermountain Healthcare sedation score; LOC, level of consciousness; Rancho scale, a cognitive functioning score; RASS, Richmond Agitation Sedation Scale; RR, respiratory rate; SBP, systolic blood pressure.

Patient impact

There were no differences in average patient age, gender, average Charlson score, or length of stay for the patients on unit A and unit B between the intervention and pre-intervention years (table 4). However, the patients on unit A were significantly older, had significantly higher Charlson comorbidity scores, and were significantly more often female than patients on unit B. Patients on unit A also had a significant increase in length of stay during the intervention year compared to the pre-intervention year and total hospital costs were significantly higher before the Bonferroni correction was applied. During the intervention year on unit A, more patients were transferred to an ICU (163 (5.1%) of 3189 compared to 146 (4.1%) of 3423; p = 0.23), there were significantly more MET calls (60 vs 29; p = 0.0008), and significantly fewer patients died (84 (2.6%) vs 125 (3.7%); p = 0.044) compared to the pre-intervention year. While there were more MET calls (21 vs 11) on unit B, no significant differences were found between the intervention and pre-intervention periods. The average physiologic deterioration scores for the patients who died were significantly higher than those who did not die (p = 0.002), but no significant difference was found between the average physiologic deterioration scores of patients transferred to the ICU.

Table 4:

Comparison of patients and outcomes in unit A and unit B during the intervention and pre-intervention years

Category Unit A
intervention 
Unit A
pre-intervention 
Unit B
intervention 
Unit B
pre-intervention 
Patients 3189 3423 3100 3169 
Average age 59.9* 60.5* 51.3 51.1 
Female, % 54* 55* 46 46 
Average Charlson comorbidity (range) 2.04 (0–12)* 2.07 (0–13)* 0.68 (0–13) 0.76 (0–12) 
Average LOS, days 4.9** 4.4 4.3 4.1 
Average cost of hospitalization, $ 12 813*** 11 509 15 929 15 700 
MET calls 60# 29 21 11 
Transferred to ICU (%) 163 (5.1) 146 (4.3) 72 (2.3) 71 (2.2) 
Died (%) 84 (2.6)## 125 (3.7) 19 (0.6) 17 (0.5) 
Average physiologic deterioration score (range), died 5.2 (4–16)* – 7.0 (4–11)* – 
Average physiologic deterioration score (range), did not die 4.5 (4–8) – 4.7 (4–11) – 
Average physiologic deterioration score (range), transferred to ICU 4.6 (4–8) – 5.0 (4–7) – 
Average physiologic deterioration score (range), not transferred to ICU 4.6 (4–16) – 5.0 (4–11) – 
Category Unit A
intervention 
Unit A
pre-intervention 
Unit B
intervention 
Unit B
pre-intervention 
Patients 3189 3423 3100 3169 
Average age 59.9* 60.5* 51.3 51.1 
Female, % 54* 55* 46 46 
Average Charlson comorbidity (range) 2.04 (0–12)* 2.07 (0–13)* 0.68 (0–13) 0.76 (0–12) 
Average LOS, days 4.9** 4.4 4.3 4.1 
Average cost of hospitalization, $ 12 813*** 11 509 15 929 15 700 
MET calls 60# 29 21 11 
Transferred to ICU (%) 163 (5.1) 146 (4.3) 72 (2.3) 71 (2.2) 
Died (%) 84 (2.6)## 125 (3.7) 19 (0.6) 17 (0.5) 
Average physiologic deterioration score (range), died 5.2 (4–16)* – 7.0 (4–11)* – 
Average physiologic deterioration score (range), did not die 4.5 (4–8) – 4.7 (4–11) – 
Average physiologic deterioration score (range), transferred to ICU 4.6 (4–8) – 5.0 (4–7) – 
Average physiologic deterioration score (range), not transferred to ICU 4.6 (4–16) – 5.0 (4–11) – 

*p = 0.0002, unit A vs unit B both years, average score between patients who died and did not die; **p = 0.002, unit A intervention vs unit A pre-intervention; ***p = 0.076, unit A intervention vs unit A pre-intervention; #p = 0.008, unit A intervention vs unit A pre-intervention; ##p = 0.04, unit A intervention vs unit A pre-intervention (Bonferroni correction applied to all tests of significance when uncorrected p values were less than 0.05).

ICU, intensive care unit; MET, medical emergency team.

The primary discharge diagnosis ICD-9 codes during the intervention and pre-intervention years were very similar for unit A and unit B. Ten of the top 11 discharge diagnoses were the same in unit A and nine of 11 were the same in unit B. Codes for sepsis were included for 22% of the patients on unit A. Of interest, 33% of the physiologic deterioration alerts during the intervention years were for patients with a primary or secondary ICD-9 code for sepsis and 91% of those were on unit A. Six of the 11 top discharge codes for patients with physiologic deterioration alerts were the same as for the total number of patients on unit A, whereas only two matched on unit B. However, on unit A, we did not find a significant decrease in mortality for sepsis patients between the intervention and pre-intervention years (p = 0.86), while there was a significant decrease in mortality for patients without sepsis (p = 0.02).

Unit A MET simulation

Of the 3189 patients on unit A from July 2, 2012 to July 1, 2013, there were 1280 (40%) during the 5 months before the MET simulation and 1909 (60%) during the 7 months afterward. Before the MET simulation, 69 (5.4%) patients were transferred to an ICU compared to 94 (4.9%) (p = 0.61), 31 (2.4%) died compared to 52 (2.7%) (p = 0.68), and 22 (1.7%) had MET calls compared to 38 (2.0%) (p = 0.67). The significant increase in MET calls and significant decrease in mortality on unit A does not appear to be attributed to the additional MET simulation training.

Discussion

There appears to be variability in the ability of healthcare institutions to detect patients who experience physiologic deterioration.54 Delayed defibrillation is common and patients who are attended to within 30–60 min of physiologic deterioration have significantly lower mortality rates.9,55,56 This study found use of computerized decision support provided a way to constantly monitor patients and notify nursing staff of early physiologic deterioration, which resulted in a significant increase in appropriate MET calls and a significant decrease in mortality in the nursing unit containing older patients with multiple comorbidities.

The success of the RRS seems to be tied to nurses’ clinical judgment, ability, experience, work environment, and willingness to activate the RRS appropriately.28,57–61 In those situations, nurses describe the RRS as life-saving for their patients and report increased morale and empowerment among nurses, real-time redistribution of workload for nurses, and immediate access to expert help.62

Nurses rely on physical examination and experience to identify a deteriorating patient when in visual contact with the patient.11 Two studies report that in situ nurse training led to improved recognition and management of deteriorating patients.63,64 However, despite nurse training and the implementation of physiologic deterioration methodologies such as MEWS, use of early warning scoring methods are rarely used during routine patient care.3,64 The simulation training we reported in this study was also not found to have a clinical impact on routine patient care. Moreover, nurses do not have constant visual contact with patients and up to 26% of nurses’ responses to abnormal vital signs result in delays of between 1 and 3 h.65 The use of continuous patient monitoring and graphical vital sign display of the physiologic deterioration alerts in this study appear to help overcome these barriers of irregular visual contact with patients and the uncertainty of the patient condition. It appears that automated decision support can reliably use MEWS, or other models, to consistently monitor and identify patients with physiologic deterioration early enough to impact patient care.

The nurses on both of the units of this study reported an appreciable difference in their workflow due to the identification of patients with early physiologic deterioration. With the patient trending information contained in the graphical alerts, nurses reported they had the information they needed to quickly evaluate the patient's status, felt more confident about their assessment, and were more at ease about requesting additional help. The nurses on unit A reported they are readily sharing the information with the medical residents or hospitalists on the unit. If the physician covering unit B was not present, nurses often shared the information with nurse practitioners or physician assistants. The nurses from both units and the MET reported that the aggregated information provided by the new physiologic deterioration alerts enabled deteriorating patients to receive expedited interventions, and in some cases avoided the need to call the MET. With the current high positive predictive value and the timing of the alerts, physicians and advance practice clinicians also have an increased interest in being included in the alert follow-up.

Acute change in level of consciousness leading to a MET call may carry a greater risk of death than activation due to arrhythmias.41 When nurses find the patient unresponsive, they do not take time to chart that information before taking clinical action. While nurses called the MET based on patient level of consciousness during the pre-intervention year, we found nurse charting of patient level of consciousness was inconsistent and most often charted after the adverse event was resolved. We also found that often patient level of consciousness was charted at levels that would have initiated an alert for hospice patients. Therefore, the early alerting system we developed was generally activated by patient vital signs data rather than patient changes in level of consciousness. The respiratory rate was the second most common indicator of physiologic deterioration in this study. Others report respiratory failure to be a primary cause of emergency admission to ICUs and the respiratory rate and other vital signs are often not documented and their importance is underestimated.66

Impact of the physiologic deterioration alerts

The bedside nurses on both study units reported that they called a physician for 51% and 44% of the alerts and interventions were initiated for 59% and 52%. Moreover, due to nursing management requests, we have installed the physiologic deterioration alerts in two other units at LDS Hospital and are ramping up programming capabilities to install them at all nursing units in all 22 IH hospitals. While there was a significant increase in MET calls on unit A and an increase on unit B, the significant decrease in mortality for patients on unit A was not found on unit B. The patients on unit A were significantly older and had significantly more comorbidities compared to unit B and thus were more likely to experience physiologic deterioration as shown by significantly more patients on unit A generating physiological deterioration alerts (17% vs 9%; p<0.0002) and almost three times the number of MET calls during the intervention and pre-intervention periods (table 4). Thus, based on the results of this study, it appears the benefit of our physiologic deterioration alerts may be higher for patients who are older and have multiple comorbidities. The patients in unit B were mostly younger or otherwise healthy males needing surgery for traumatic accidents. The fact that the length of stay was significantly higher in unit A and slightly higher in unit B during the intervention year compared to the pre-intervention year may be due to critical patients getting additional interventions, generating MET calls, or being transferred to the ICU early enough to survive and be discharged alive. Thus, those patients’ length of stay would be expected to be longer and their hospital cost higher. In fact, the patients on unit A who had physiologic deterioration alerts had an average cost and length of stay of $22 928 and 4.7 days compared to $11 966 and 3.2 days for patients without alerts on unit A, and $45 333 and 4.9 days for patients with alerts compared to $15 929 and 3.2 days without alerts on unit B. Further research on the potential benefits of the physiologic deterioration alerts reported in this paper will be maintained as we continue to implement the alerts in more IH urban and rural hospitals.

Previous automated detection of physiologic deterioration

Previous studies report different electronic methods to help the early detection of physiologic deterioration. Initially, vital signs were manually entered into a personal digital assistant or the EMR and MEWS scores were calculated and sent to reports or pagers.45,47,50 Manual use of MEWS is very costly per event detected and not recommended for monitoring all patients continuously.3 Others have automated the collection of patient data and send risk scores to staff at 8–12 h intervals.44,46,48,49 During short term or retrospective analyses, some previous studies report an increase in MET calls and a decrease in non-ICU codes or their predictive models compared favorable to other scoring systems.44,48,49 To our knowledge, this is the first publication concerning a continuous real-time implementation and evaluation of nursing-accepted and endorsed alerts of physiologic deterioration. The 23-hospital MERIT study's inability to report a consistent and significant improvement in patient outcomes after the implementation of RRS may have been the result of physiologic deterioration not being noticed early enough at some hospitals.9,17,34

Limitations

This before–after, observational study reports our findings on the positive predictive value of the physiologic deterioration alerts based on nursing or MET evaluation on only two nursing units. While there were significantly more MET calls only on unit A, the opinion of the nurses and MET nurses is that a number of MET calls were avoided due to the early notification of physiologic deterioration. Thus, the sole measurement of nursing approval, the number of MET calls, ICU transfers, and mortality may not represent the only positive impacts of the physiologic deterioration alerts. Also, while primary discharge diagnosis ICD-9 codes during the intervention and pre- intervention years were very similar for unit A and unit B and no impact of the simulation training on unit A was detected or of other secular trends at the unit level, other unit changes cannot be totally ruled out.

For those alerts where bedside nurses did not fill out evaluations, the MET leads and intensive care nurses retrospectively examined the patients and reported a lower positive predictive value than the bedside nurses. Overall acceptance and appreciation of the alerts may have biased the bedside nurses’ higher evaluation. Also, while the physiologic deterioration alerts were generated during the entire intervention year for unit A and unit B, the extra burden on the bedside nurses and the MET and intensive care nurses to evaluate the alerts was terminated on April 30, 2013. We have no reason to think the data would have changed with the additional 2 months of data on unit A or 4 months on unit B.

Conclusions

The ability to monitor hospitalized patients every 5 min and provide automated detection of early physiologic deterioration resulted in a significant increase in appropriate MET calls and a significant decrease in mortality, but only in the nursing unit containing older patients with multiple comorbidities, and thus further study is warranted to detect potential confounding. Moreover, nurses reported that the graphical alerts provided the information they needed to quickly evaluate patient status, they felt more confident about their assessment, and they were more at ease about requesting additional help.

The authors wish to thank Joyce Gamble, MS, RN, CCRN, Annan Fetzer, RN, BSN, Laurie Hastings, RN, BSN, and Rachelle Reid, RN for providing the MET education and physiologic deterioration simulation on unit A during the study and Jeffrey P. Ferraro, MS, PhD and Chaitanya Kumar Mynam, MS in the Homer Warner Center for Informatics Research at Intermountain Healthcare for their help with developing the alert web page that displayed the same alert information for nursing staff contained in the email alerts.

Contributors

RSE, KGK, KJS, SH, and KVJ contributed to the initial conception and design of the study. RSE, KGK, KJS, SH, PFC, MNS, WHT, RKK, AT, CW, KVJ, and TPC contributed to the development, testing, and analysis of the computer alerting logic and final study design. RSE, KGK, KJS, and JFL contributed to the acquisition and analysis of the data. All authors contributed to interpretation of the data. RSE contributed the initial draft of the publication. All authors were involved with the editing of the publication, approval of the data, reported findings, and final version of the publication, and integrity of the work.

Funding

This work was a quality improvement project funded by Intermountain Healthcare.

Competing interests

None.

Ethics approval

Intermountain Healthcare Office of Research approved this study.

Provenance and peer review

Not commissioned; externally peer reviewed.

REFERENCES

1
Bell
MB
Konrad
D
Granath
F
et al
.
Prevalence and sensitivity of MET-criteria in a Scandinavian University Hospital
.
Resuscitation
 .
2006
;
70
:
66
73
.
2
McFarlan
SJ
Hensley
S
.
Implementation and outcomes of a rapid response team
.
J Nurs Care Qual
 .
2007
;
22
:
307
13
,
quiz 14–15
.
3
Devita
MA
Bellomo
R
Hillman
K
et al
.
Findings of the first consensus conference on medical emergency teams
.
Crit Care Med
 .
2006
;
34
:
2463
2478
.
4
Goldhill
DR
White
SA
Sumner
A
.
Physiological values and procedures in the 24 h before ICU admission from the ward
.
Anaesthesia
 .
1999
;
54
:
529
534
.
5
Hillman
KM
Bristow
PJ
Chey
T
et al
.
Antecedents to hospital deaths
.
Intern Med J
 .
2001
;
31
:
343
348
.
6
Hillman
KM
Bristow
PJ
Chey
T
et al
.
Duration of life-threatening antecedents prior to intensive care admission
.
Intensive Care Med
 .
2002
;
28
:
1629
1634
.
7
Schein
RM
Hazday
N
Pena
M
et al
.
Clinical antecedents to in-hospital cardiopulmonary arrest
.
Chest
 .
1990
;
98
:
1388
1392
.
8
Buist
MD
Jarmolowski
E
Burton
PR
et al
.
Recognising clinical instability in hospital patients before cardiac arrest or unplanned admission to intensive care. A pilot study in a tertiary-care hospital
.
Med J Aust
 .
1999
;
171
:
22
25
.
9
Buist
M
.
The rapid response team paradox: why doesn't anyone call for help?
Crit Care Med
 .
2008
;
36
:
634
636
.
10
Kause
J
Smith
G
Prytherch
D
et al
.
A comparison of antecedents to cardiac arrests, deaths and emergency intensive care admissions in Australia and New Zealand, and the United Kingdom--the ACADEMIA study
.
Resuscitation
 .
2004
;
62
:
275
282
.
11
Cioffi
J
.
Recognition of patients who require emergency assistance: a descriptive study
.
Heart Lung
 .
2000
;
29
:
262
268
.
12
Ott
LK
Pinsky
MR
Hoffman
LA
et al
.
Medical emergency team calls in the radiology department: patient characteristics and outcomes
.
BMJ Qual Saf
 .
2012
;
21
:
509
518
.
13
Bruckel
J
.
Evidence-based medicine and rapid response team implementation
.
McGill J Med
 .
2006
;
9
:
5
7
.
14
Odell
M
Rechner
IJ
Kapila
A
et al
.
The effect of a critical care outreach service and an early warning scoring system on respiratory rate recording on the general wards
.
Resuscitation
 .
2007
;
74
:
470
475
.
15
Donaldson
N
Shapiro
S
Scott
M
et al
.
Leading successful rapid response teams: a multisite implementation evaluation
.
J Nurs Adm
 .
2009
;
39
:
176
181
.
16
Galhotra
S
Scholle
CC
Dew
MA
et al
.
Medical emergency teams: a strategy for improving patient care and nursing work environments
.
J Adv Nurs
 .
2006
;
55
:
180
187
.
17
Moldenhauer
K
Sabel
A
Chu
ES
et al
.
Clinical triggers: an alternative to a rapid response team
.
Jt Comm J Qual Patient Saf
 .
2009
;
35
:
164
174
.
18
Morris
DS
Schweickert
W
Holena
D
et al
.
Differences in outcomes between ICU attending and senior resident physician led medical emergency team responses
.
Resuscitation
 .
2012
;
83
:
1434
1437
.
19
Dacey
MJ
Mirza
ER
Wilcox
V
et al
.
The effect of a rapid response team on major clinical outcome measures in a community hospital
.
Crit Care Med
 .
2007
;
35
:
2076
2082
.
20
Howell
MD
Ngo
L
Folcarelli
P
et al
.
Sustained effectiveness of a primary-team-based rapid response system
.
Crit Care Med
 .
2012
;
40
:
2562
2568
.
21
Hueckel
RM
Turi
JL
Cheifetz
IM
et al
.
Beyond rapid response teams: instituting a “Rover Team” improves the management of at-risk patients, facilitates proactive interventions, and improves outcomes
. In:
Henriksen
K
Battles
JB
Keyes
MA
, eds.
Advances in Patient Safety: New Directions and Alternative Approaches (Vol 3: Performance and Tools)
 .
Rockville, MD: Agency for Healthcare Research and Quality
;
2008
.
22
Psirides
A
Hill
J
Hurford
S
.
A review of rapid response team activation parameters in New Zealand hospitals
.
Resuscitation
 .
2013
;
84
:
1040
1044
.
23
Sharek
PJ
Parast
LM
Leong
K
et al
.
Effect of a rapid response team on hospital-wide mortality and code rates outside the ICU in a children's Hospital
.
JAMA
 .
2007
;
298
:
2267
2274
.
24
Subbe
CP
Gao
H
Harrison
DA
.
Reproducibility of physiological track-and-trigger warning systems for identifying at-risk patients on the ward
.
Intensive Care Med
 .
2007
;
33
:
619
624
.
25
Bellomo
R
Goldsmith
D
Uchino
S
et al
.
A prospective before-and-after trial of a medical emergency team
.
Med J Aust
 .
2003
;
179
:
283
287
.
26
Pittard
AJ
.
Out of our reach? Assessing the impact of introducing a critical care outreach service
.
Anaesthesia
 .
2003
;
58
:
882
885
.
27
Al-Qahtani
S
Al-Dorzi
HM
Tamim
HM
et al
.
Impact of an intensivist-led multidisciplinary extended rapid response team on hospital-wide cardiopulmonary arrests and mortality
.
Crit Care Med
 .
2013
;
41
:
506
517
.
28
Beitler
JR
Link
N
Bails
DB
et al
.
Reduction in hospital-wide mortality after implementation of a rapid response team: a long-term cohort study
.
Crit Care
 .
2011
;
15
:
R269
.
29
Bertaut
Y
Campbell
A
Goodlett
D
.
Implementing a rapid-response team using a nurse-to-nurse consult approach
.
J Vasc Nurs
 .
2008
;
26
:
37
42
.
30
Gerdik
C
Vallish
RO
Miles
K
et al
.
Successful implementation of a family and patient activated rapid response team in an adult level 1 trauma center
.
Resuscitation
 .
2010
;
81
:
1676
1681
.
31
Jaderling
G
Bell
M
Martling
CR
et al
.
ICU admittance by a rapid response team versus conventional admittance, characteristics, and outcome
.
Crit Care Med
 .
2013
;
41
:
725
731
.
32
King
E
Horvath
R
Shulkin
DJ
.
Establishing a rapid response team (RRT) in an academic hospital: one year's experience
.
J Hosp Med
 .
2006
;
1
:
296
305
.
33
Moon
A
Cosgrove
JF
Lea
D
et al
.
An eight year audit before and after the introduction of modified early warning score (MEWS) charts, of patients admitted to a tertiary referral intensive care unit after CPR
.
Resuscitation
 .
2011
;
82
:
150
154
.
34
Hillman
K
Chen
J
Cretikos
M
et al
.
Introduction of the medical emergency team (MET) system: a cluster- randomised controlled trial
.
Lancet
 .
2005
;
365
:
2091
2097
.
35
Butcher
BW
Vittinghoff
E
Maselli
J
et al
.
Impact of proactive rounding by a rapid response team on patient outcomes at an academic medical center
.
J Hosp Med
 .
2013
;
8
:
7
12
.
36
Chan
PS
Khalid
A
Longmore
LS
et al
.
Hospital-wide code rates and mortality before and after implementation of a rapid response team
.
JAMA
 .
2008
;
300
:
2506
2513
.
37
Rothberg
MB
Belforti
R
Fitzgerald
J
et al
.
Four years’ experience with a hospitalist-led medical emergency team: an interrupted time series
.
J Hosp Med
 .
2012
;
7
:
98
103
.
38
Shah
SK
Cardenas
VJ
Jr
Kuo
YF
et al
.
Rapid response team in an academic institution: does it make a difference?
Chest
 .
2011
;
139
:
1361
1367
.
39
Vazquez
R
Gheorghe
C
Grigoriyan
A
et al
.
Enhanced end-of-life care associated with deploying a rapid response team: a pilot study
.
J Hos Med
 .
2009
;
4
:
449
452
.
40
Burch
VC
Tarr
G
Morroni
C
.
Modified early warning score predicts the need for hospital admission and inhospital mortality
.
Emerg Med J
 .
2008
;
25
:
674
678
.
41
Downey
AW
Quach
JL
Haase
M
et al
.
Characteristics and outcomes of patients receiving a medical emergency team review for acute change in conscious state or arrhythmias
.
Crit Care Med
 .
2008
;
36
:
477
481
.
42
Morgan
RJM
.
An early warning scoring system for detecting developing critical illness
.
Clin Intensive Care
 .
1997
;
8
:
100
.
43
Subbe
CP
Kruger
M
Rutherford
P
et al
.
Validation of a modified early warning score in medical admissions
.
QJM
 .
2001
;
94
:
521
526
.
44
Tarassenko
L
Hann
A
Young
D
.
Integrated monitoring and analysis for early warning of patient deterioration
.
Br J Anaesth
 .
2006
;
97
:
64
68
.
45
Jones
S
Mullally
M
Ingleby
S
et al
.
Bedside electronic capture of clinical observations and automated clinical alerts to improve compliance with an Early Warning Score protocol
.
Crit Care Resusc
 .
2011
;
13
:
83
88
.
46
Whittington
J
White
R
Haig
KM
et al
.
Using an automated risk assessment report to identify patients at risk for clinical deterioration
.
Jt Comm J Qual Patient Saf
 .
2007
;
33
:
569
574
.
47
Smith
GB
Prytherch
DR
Schmidt
P
et al
.
Hospital-wide physiological surveillance-a new approach to the early identification and management of the sick patient
.
Resuscitation
 .
2006
;
71
:
19
28
.
48
Escobar
GJ
LaGuardia
JC
Turk
BJ
et al
.
Early detection of impending physiologic deterioration among patients who are not in intensive care: development of predictive models using data from an automated electronic medical record
.
J Hosp Med
 .
2012
;
7
:
388
395
.
49
Kho
A
Rotz
D
Alrahi
K
et al
.
Utility of commonly captured data from an EHR to identify hospitalized patients at risk for clinical deterioration
.
AMIA Annu Symp Proc
 .
2007
:
404
408
.
50
Albert
BL
Huesman
L
.
Development of a modified early warning score using the electronic medical record
.
Dimens Crit Care Nurs
 .
2011
;
30
:
283
292
.
51
Pryor
TA
Gardner
RM
Clayton
PD
et al
.
The HELP system
.
J Med Syst
 .
1983
;
7
:
87
102
.
52
Pryor
TA
.
Computerized nurse charting
.
Int J Clin Monitor Comput
 .
1989
;
6
:
173
179
.
53
Gardner
RM
Hawley
WL
East
TD
et al
.
Real time data acquisition: experience with the Medical Information Bus (MIB)
.
Proceedings/the Annual Symposium on Computer Application [sic] in Medical Care Symposium on Computer Applications in Medical Care
 
1991
:
813
817
.
54
Galhotra
S
DeVita
MA
Simmons
RL
et al
.
Impact of patient monitoring on the diurnal pattern of medical emergency team activation
.
Crit Care Med
 .
2006
;
34
:
1700
1706
.
55
Kumar
A
Roberts
D
Wood
KE
et al
.
Duration of hypotension before initiation of effective antimicrobial therapy is the critical determinant of survival in human septic shock
.
Crit Care Med
 .
2006
;
34
:
1589
1596
.
56
Chan
PS
Krumholz
HM
Nichol
G
et al
.
Delayed time to defibrillation after in-hospital cardiac arrest
.
N Engl J Med
 .
2008
;
358
:
9
17
.
57
Metcalf
R
Scott
S
Ridgway
M
et al
.
Rapid response team approach to staff satisfaction
.
Orthop Nurs
 .
2008
;
27
:
266
71
;
quiz 72–73
.
58
Chen
LM
Nallamothu
BK
Spertus
JA
et al
.
Association between a hospital's rate of cardiac arrest incidence and cardiac arrest survival
.
JAMA Intern Med
 .
2013
;173:1186–1195
.
59
Williams
DJ
Newman
A
Jones
C
et al
.
Nurses’ perceptions of how rapid response teams affect the nurse, team, and system
.
J Nurs Care Qual
 .
2011
;
26
:
265
272
.
60
Wynn
JD
Engelke
MK
Swanson
M
.
The front line of patient safety: staff nurses and rapid response team calls
.
Qual Manag Health Care
 .
2009
;
18
:
40
47
.
61
Daffurn
K
Lee
A
Hillman
KM
et al
.
Do nurses know when to summon emergency assistance?
Intensive Crit Care Nurs
 .
1994
;
10
:
115
120
.
62
Benin
AL
Borgstrom
CP
Jenq
GY
et al
.
Defining impact of a rapid response team: qualitative study with nurses, physicians and hospital administrators
.
BMJ Qual Saf
 .
2012
;
21
:
391
398
.
63
Theilen
U
Leonard
P
Jones
P
et al
.
Regular in situ simulation training of paediatric medical emergency team improves hospital response to deteriorating patients
.
Resuscitation
 .
2013
;
84
:
218
222
.
64
Ludikhuize
J
de Jonge
E
Goossens
A
.
Measuring adherence among nurses one year after training in applying the Modified Early Warning Score and Situation-Background-Assessment-Recommendation instruments
.
Resuscitation
 .
2011
;
82
:
1428
1433
.
65
Thompson
C
Bucknall
T
Estabrookes
CA
et al
.
Nurses’ critical event risk assessments: a judgement analysis
.
J Clin Nurs
 .
2009
;
18
:
601
612
.
66
Jonsson
T
Jonsdottir
H
Moller
AD
et al
.
Nursing documentation prior to emergency admissions to the intensive care unit
.
Nurs Crit Care
 .
2011
;
16
:
164
169
.

Supplementary data

Comments

0 Comments