http://orcid.org/0000-0002-3643-0403
Abstract
Many hospital systems in the United States report injurious inpatient falls using the National Database of Nursing Quality Indicators categories: None, Minor, Moderate, Major, and Death. The Major category is imprecise, including injuries ranging from a wrist fracture to potentially fatal subdural hematoma. The purpose of this project was to refine the Major injury classification to derive a valid and reliable categorization of the types and severities of Major inpatient fall-related injuries.
Based on published literature and ranking of injurious fall incident reports (n = 85) from a large Academic Medical Center, we divided the National Database of Nursing Quality Indicators Major category into three subcategories: Major A—injuries that caused temporary functional impairment (eg, wrist fracture), major facial injury without internal injury (eg, nasal bone fracture), or disruption of a surgical wound; Major B—injuries that caused long-term functional impairment or had the potential risk of increased mortality (eg, multiple rib fractures); and Major C—injuries that had a well-established risk of mortality (eg, hip fracture). Based on the literature and expert opinion, our research team reached consensus on an administration manual to promote accurate classification of Major injuries into one of the three subcategories.
The team tested and validated each of the categories which resulted in excellent interrater reliability (kappa = .96). Of the Major injuries, the distribution of Major A, B, and C was 40.3%, 16.1%, and 43.6%, respectively.
These subcategories enhance the National Database of Nursing Quality Indicators categorization. Using the administration manual, trained personnel can classify injurious fall severity with excellent reliability.
Falls represent a major safety issue in hospitals; the Agency for Healthcare Research and Quality estimated between 700,000 and 1,000,000 inpatient falls in the United States annually (1) with 30% resulting in injury (2,3). Major injuries from falls are relatively rare, accounting for roughly 4–12% of injurious falls (3–10). Risk of injury from falls increases as people age, and the consequences are significant (11). Injuries such as subdural hematoma and hip fractures can cause long-term functional impairment and even death (6). On average, injurious falls increase hospital stays by 6–12 days (12–15) and cost an additional $7,900–$17,099 (2019 USD) (14,16). The cost of serious episodes of injury ranges from $19,376 to $32,215 (2019 USD) (16–18). Approximately 78% of inpatient falls are preventable (19) and fall-related injuries are not reimbursable by the Centers for Medicare and Medicaid Services (20).
The National Database of Nursing Quality Indicators (NDNQI) classification system is used to quantify the severity of injurious inpatient falls in one third of large hospital systems in the United States (21,22). This typology consists of five categories ranging from no injury to death because of the fall (Table 1). Such standardized guidelines for assessing the severity and types of injurious falls are imperative to informing the development of appropriate education, risk assessment, and fall prevention plans and assessing their impact (21).
National Database of Nursing Quality Indicators (NDNQI) Injury Severity Classifications (21)
| NDNQI Scale | NDNQI Injury Classification | Definition |
|---|---|---|
| 0 | None | No injuries resulting from the fall and no confirmation of injury by any post fall test. |
| 1 | Minor | Fall resulted in application of a dressing, ice, cleaning of a wound, limb elevation, topical medication, bruise, or abrasion. |
| 2 | Moderate | Fall resulted in suturing, application of steri-strips/skin glue, and splinting of muscle/joint strain. |
| 3 | Major | Fall resulted in surgery, casting, traction, required consultation for neurological (basilar skull fracture, small subdural hematoma) or internal injury (rib fracture, liver laceration) or patients with coagulopathy who receive blood products as a result of the fall. |
| 4 | Death | Death as a result of injuries from the fall and not the physiological events causing the fall. |
| NDNQI Scale | NDNQI Injury Classification | Definition |
|---|---|---|
| 0 | None | No injuries resulting from the fall and no confirmation of injury by any post fall test. |
| 1 | Minor | Fall resulted in application of a dressing, ice, cleaning of a wound, limb elevation, topical medication, bruise, or abrasion. |
| 2 | Moderate | Fall resulted in suturing, application of steri-strips/skin glue, and splinting of muscle/joint strain. |
| 3 | Major | Fall resulted in surgery, casting, traction, required consultation for neurological (basilar skull fracture, small subdural hematoma) or internal injury (rib fracture, liver laceration) or patients with coagulopathy who receive blood products as a result of the fall. |
| 4 | Death | Death as a result of injuries from the fall and not the physiological events causing the fall. |
National Database of Nursing Quality Indicators (NDNQI) Injury Severity Classifications (21)
| NDNQI Scale | NDNQI Injury Classification | Definition |
|---|---|---|
| 0 | None | No injuries resulting from the fall and no confirmation of injury by any post fall test. |
| 1 | Minor | Fall resulted in application of a dressing, ice, cleaning of a wound, limb elevation, topical medication, bruise, or abrasion. |
| 2 | Moderate | Fall resulted in suturing, application of steri-strips/skin glue, and splinting of muscle/joint strain. |
| 3 | Major | Fall resulted in surgery, casting, traction, required consultation for neurological (basilar skull fracture, small subdural hematoma) or internal injury (rib fracture, liver laceration) or patients with coagulopathy who receive blood products as a result of the fall. |
| 4 | Death | Death as a result of injuries from the fall and not the physiological events causing the fall. |
| NDNQI Scale | NDNQI Injury Classification | Definition |
|---|---|---|
| 0 | None | No injuries resulting from the fall and no confirmation of injury by any post fall test. |
| 1 | Minor | Fall resulted in application of a dressing, ice, cleaning of a wound, limb elevation, topical medication, bruise, or abrasion. |
| 2 | Moderate | Fall resulted in suturing, application of steri-strips/skin glue, and splinting of muscle/joint strain. |
| 3 | Major | Fall resulted in surgery, casting, traction, required consultation for neurological (basilar skull fracture, small subdural hematoma) or internal injury (rib fracture, liver laceration) or patients with coagulopathy who receive blood products as a result of the fall. |
| 4 | Death | Death as a result of injuries from the fall and not the physiological events causing the fall. |
A review of the injurious fall incident reports from our medical center and the NDNQI categories and definitions revealed the wide range of injuries and treatments included in the NDNQI Major injury classification: from casting for a wrist fracture to surgery for a life-threatening subdural hematoma. Since treatment of a given injury can be subjective and differs per individual provider and geographic area (23), classifying an injury based on treatment is not objective and reproducible, limiting this method of classification. In 2016, Garrard and colleagues (21) reported on the reliability and validity of the NDNQI falls with injury measure. The authors concluded that the measure was both reliable and valid; however, the results revealed that raters, familiar with NDNQI definitions and guidelines, had difficulty correctly classifying Moderate and Major injuries (21). The authors attributed these findings to three potential issues: confusion over the definitions (of Moderate and Major injuries), ambiguity of incident reports, and bias from either the injured patient or the fall evaluator’s perspectives (21). Since Moderate and Major injuries from falls are rare, and the Garrard study utilized 15 fall scenarios, the authors acknowledged that their analysis would have been strengthened by including an increased variety of incident reports representing a more complete picture of fall events occurring in hospitals. Garrard and colleagues called for increased education and training around Moderate and Major injuries. Given the potentially devastating impact of a serious injury caused by a fall, the purpose of this project was to refine the Major injury classification to derive a reliable categorization of the types and severities of major fall-related injuries by carrying out a series of five steps (Figure 1).
Injurious falls classification methods.
Injurious falls classification methods.
Methods and Results
Step 1: Literature Review
Methods
We systematically searched the literature to identify all injuries associated with inpatient falls. We combined the concepts of falls, the hospital setting, and injuries in Medline and CINAHL databases (Supplementary Appendix A, see search 1).This provided objective criteria for determining the severity of injuries from falls classified as Major. We identified evidence on the types of fall-related injuries that could be fatal and researched the Standardized Mortality Ratios associated with various injuries. Additional information on the varying consequences of these injuries was needed to create precise Major injury subcategories.
Two researchers independently examined the literature to identify supplementary details on the level of harm patients experienced related to functional impairment, morbidity, mortality, and compromised quality of life from injurious falls. We searched combinations of terms in the CINAHL database and combined the concepts of falls, the hospital/inpatient setting, and resulting consequences of specific injuries identified in the previous literature review (see Supplementary Appendix A, search 2). We searched selected publications’ references and “similar articles” in PubMed for more articles of interest (Supplementary Appendix A, see search 3). To address conflicting information, further investigation of the sequelae of specific injuries was performed using MeSH indexing of key terms in PubMed, UpToDate, and medical textbooks through Access Medicine and DynaMedPlus (Supplementary Appendix A, see search 4).
Results of Step 1
Published literature on injurious falls has focused on the severity of serious head injuries and hip fractures. The limited agreement on the severity of many other injuries that occur from low-impact falls is notable. For example, there is conflicting evidence on the severity of wrist fractures (24–26). Some studies found wrist fractures were associated with increased mortality, particularly in women and older adults (26), while others suggested patients with wrist fractures generally regain full functional status and quality of life within 6 months (25).
Step 2: Collect and Analyze Injurious Fall Incident Reports
Methods
Our team obtained all inpatient incident reports (n = 85) from one Academic Medical Center between January 1, 2010, and December 31, 2015, that had been rated as a Moderate or Major fall-related injury per NDNQI definitions. While the primary interest was in refining the Major injury category, we reviewed the Moderate falls for two reasons: to capture any Major injuries that had been misclassified and to learn if the proposed subcategories were aligned with the Major injuries under the NDNQI or if there was overlap with the NDNQI’s Moderate category (21). One investigator performed chart reviews in the electronic health record (EHR) to obtain information missing in the incident report (eg, test results).
Results of Step 2
Approximately 40% of those incident reports that classified injuries as Moderate and Major were incomplete, failing to include imaging results or having incomplete information regarding injury severity. This finding was consistent with the literature; for example, needing to improve the capture of fall events in hospitals—even when the overall NDNQI rating had been updated based on this information (27).
Step 3: Use Sample Cases to Develop Preliminary Major Falls Categories and Operational Definitions
Methods
Three researchers independently ranked all 85 injuries in the Moderate and Major incident reports in order of severity. Initial severity was determined by using the available literature on recovery time, expected return to baseline functioning, impact of an injury on daily functioning and quality of life, and association with increased mortality. Disagreements in classification that could not be mapped to the literature for initial validation were brought to the larger group of co-investigators for discussion. Agreements/disagreements were highlighted during meetings to confirm rationales for concurrence, discuss reasons for disagreement, and seek group consensus. When consensus could not be reached, we consulted experts and reexamined the literature on those injuries. Operational definitions were updated to correspond with consensus decisions. We continued this iterative process until all team members agreed on each subcategory’s operational definition.
Results of Step 3
The Major injury category was composed of injuries causing short-term impairment, long-term impairment or having a strong association with mortality, forming three distinct subcategories. The initial operational definitions for these subcategories and classified injuries from incident reports clustered under the following categories: Moderate (following the NDNQI definition in Table 1), Major A, Major B, or Major C, or unable to determine. The operational definitions for each of the major injury subcategories were as follows: Major A—injuries that caused temporary functional impairment; Major B—injuries that caused long-term functional impairment or were shown in some studies to be associated with increased mortality; and Major C—injuries that were consistently associated with a risk of mortality in the literature.
Step 4: Test and Iteratively Refine Operational Definitions With Experts
Methods
To test the subcategories, we created a rating form with a companion Injurious Falls Classification Administration Manual (IFCAM) to assure consistency in rating and used the Research Electronic Data Capture platform (28), a secure web application for designing surveys and databases. Ten nurse and two physician co-investigators pilot tested the IFCAM with a subset of de-identified incident reports for all Moderate and Major injuries from falls (n = 40) occurring at a large Academic Medical Center between January 1, 2010, and December 31, 2015. Four rounds of 10 cases were assigned via Research Electronic Data Capture for independent classification. Interrater reliability of the modified NDNQI assessment tool was measured by calculating the weighted kappa coefficient for a pair of clinician investigators who rated the same case. We estimated a common kappa across all pairs as well as a 95% confidence interval using a generalized estimating equations approach (29). We estimated the probability of agreement between a pair of raters using generalized estimating equations clustering by case. Statistical analyses for the interclass coefficient were conducted using SAS software, version 9.4. Weighted kappa and the probability of agreement between a pair of raters were calculated after each round. For injuries where our team had difficulty reaching agreement without considering treatment, we elicited the opinions of experts and reexamined the literature to inform further discussion and achieve consensus. Disagreements were discussed after each round until the group reached a consensus on the severity of all injuries. We updated the operational definitions of Major A, B, and C, example injuries in each subgroup, and the IFCAM based on discussions and findings.
Results of Step 4
Our team had difficulty reaching agreement on several injuries without considering treatment. These injuries included wound dehiscence/disruption of a surgical wound post-joint replacement surgery, head/face trauma with negative imaging results (eg, ruptured globe, occipital hematoma), displaced versus nondisplaced fractures, and a bilateral rectus sheath hematoma. This process revealed the amount of data and optimal timing of injury assessment needed for reliable classification. Once the process of 12 co-investigators rating 10 cases was repeated four times, we achieved excellent interrater reliability (kappa = .93, 95% CI .89 to .98). The probability of pair agreement for these cases was .87 (.77–.93).
Step 5: Validate Operational Definitions With a New Sample of Fall Injury Cases
Methods
Once investigators agreed with the operational definitions and the effectiveness of the IFCAM to facilitate classification of injury severity, we collected incident reports for all Moderate and Major fall injuries between January 1, 2016, and September 30, 2018, from three large healthcare systems (12 hospitals in Massachusetts and New York). Three team members, not involved in the case classification, performed chart reviews in the EHR to identify missing information in the incident reports, such as imaging results. Reviews of imaging results and notes in the EHR were limited to patients’ length of stay. Co-investigators, blinded to the study site and unit, used the Research Electronic Data Capture rating form and the IFCAM to independently classify 240 de-identified incident reports (six pairs of two investigators independently rated a set of 40 cases) of all injuries rated Moderate or Major. Disagreements in severity ratings were discussed as a group and were used to inform where additional “tricky” case examples were needed as examples in the IFCAM. Interrater disagreements where a consensus could not be reached with further group discussion were resolved by an arbiter with expertise in event classification. Fifteen additional cases were discarded as they did not contain enough information to confirm a Moderate or Major injury. Chart reviews were performed to locate information missing in the incident reports. Eight cases were removed prior to analysis because they did not contain enough information to confirm a Moderate or Major injury; for example, the patient was discharged without imaging results recorded in EHR. Interrater reliability for injurious fall incident report (n = 232) severity classification was calculated using mean kappa scores between reviewers.
Results of Step 5
Chart review was required in approximately 40% of cases to locate information missing in the incident reports (as found with the 2010–2015 incident reports). A total of 217 injuries remained for further analysis. Three distinct clusters of severity emerged from our data collection. We finalized subcategory operational definitions (Table 2), example injuries, and the IFCAM (Supplementary Appendix B).
NDNQI Major Injury Definition and Proposed Major Injury Subcategories With Examples
| NDNQI Classification (21) | Proposed Major Injury Subcategories | Subcategory Examples |
|---|---|---|
| NDNQI Major: resulted in surgery, casting, traction, required consultation for neurological (basilar skull fracture, small subdural hematoma) or internal injury (rib fracture, small liver laceration) or patients with coagulopathy who receive blood products as a result of the fall | Major A: Temporary functional impairment (including ankle, foot/toes, wrist, forearm, hand/fingers, upper arm [excluding proximal humerus fractures]); or major facial injury without visual impairment (including broken teeth, missing teeth, nasal bone fracture); or disruption of surgical wound | • Dislocated shoulder • Fracture of 1–2 ribs • Distal radius fracture • Broken tooth |
| Major B: Long-term functional impairment or potential for impaired survival (including vertebral, pelvic, multiple rib (≥3), distal femur, proximal tibia, and proximal humerus fractures; and blunt abdominal injuries) | • Femur fracture with pain near the knee • Patellar fracture • Displaced or nondisplaced fracture of the sacrum • Dislocated total hip replacement | |
| Major C: Potentially fatal (major head injury [including severe traumatic brain injury such as subdural hematoma, intracranial hemorrhage, subarachnoid hemorrhage]; and hip fracture [including femoral neck and proximal femur fractures]) | • Basilar skull fracture • Acute epidural hematoma • Displaced or nondisplaced proximal femur fracture |
| NDNQI Classification (21) | Proposed Major Injury Subcategories | Subcategory Examples |
|---|---|---|
| NDNQI Major: resulted in surgery, casting, traction, required consultation for neurological (basilar skull fracture, small subdural hematoma) or internal injury (rib fracture, small liver laceration) or patients with coagulopathy who receive blood products as a result of the fall | Major A: Temporary functional impairment (including ankle, foot/toes, wrist, forearm, hand/fingers, upper arm [excluding proximal humerus fractures]); or major facial injury without visual impairment (including broken teeth, missing teeth, nasal bone fracture); or disruption of surgical wound | • Dislocated shoulder • Fracture of 1–2 ribs • Distal radius fracture • Broken tooth |
| Major B: Long-term functional impairment or potential for impaired survival (including vertebral, pelvic, multiple rib (≥3), distal femur, proximal tibia, and proximal humerus fractures; and blunt abdominal injuries) | • Femur fracture with pain near the knee • Patellar fracture • Displaced or nondisplaced fracture of the sacrum • Dislocated total hip replacement | |
| Major C: Potentially fatal (major head injury [including severe traumatic brain injury such as subdural hematoma, intracranial hemorrhage, subarachnoid hemorrhage]; and hip fracture [including femoral neck and proximal femur fractures]) | • Basilar skull fracture • Acute epidural hematoma • Displaced or nondisplaced proximal femur fracture |
NDNQI Major Injury Definition and Proposed Major Injury Subcategories With Examples
| NDNQI Classification (21) | Proposed Major Injury Subcategories | Subcategory Examples |
|---|---|---|
| NDNQI Major: resulted in surgery, casting, traction, required consultation for neurological (basilar skull fracture, small subdural hematoma) or internal injury (rib fracture, small liver laceration) or patients with coagulopathy who receive blood products as a result of the fall | Major A: Temporary functional impairment (including ankle, foot/toes, wrist, forearm, hand/fingers, upper arm [excluding proximal humerus fractures]); or major facial injury without visual impairment (including broken teeth, missing teeth, nasal bone fracture); or disruption of surgical wound | • Dislocated shoulder • Fracture of 1–2 ribs • Distal radius fracture • Broken tooth |
| Major B: Long-term functional impairment or potential for impaired survival (including vertebral, pelvic, multiple rib (≥3), distal femur, proximal tibia, and proximal humerus fractures; and blunt abdominal injuries) | • Femur fracture with pain near the knee • Patellar fracture • Displaced or nondisplaced fracture of the sacrum • Dislocated total hip replacement | |
| Major C: Potentially fatal (major head injury [including severe traumatic brain injury such as subdural hematoma, intracranial hemorrhage, subarachnoid hemorrhage]; and hip fracture [including femoral neck and proximal femur fractures]) | • Basilar skull fracture • Acute epidural hematoma • Displaced or nondisplaced proximal femur fracture |
| NDNQI Classification (21) | Proposed Major Injury Subcategories | Subcategory Examples |
|---|---|---|
| NDNQI Major: resulted in surgery, casting, traction, required consultation for neurological (basilar skull fracture, small subdural hematoma) or internal injury (rib fracture, small liver laceration) or patients with coagulopathy who receive blood products as a result of the fall | Major A: Temporary functional impairment (including ankle, foot/toes, wrist, forearm, hand/fingers, upper arm [excluding proximal humerus fractures]); or major facial injury without visual impairment (including broken teeth, missing teeth, nasal bone fracture); or disruption of surgical wound | • Dislocated shoulder • Fracture of 1–2 ribs • Distal radius fracture • Broken tooth |
| Major B: Long-term functional impairment or potential for impaired survival (including vertebral, pelvic, multiple rib (≥3), distal femur, proximal tibia, and proximal humerus fractures; and blunt abdominal injuries) | • Femur fracture with pain near the knee • Patellar fracture • Displaced or nondisplaced fracture of the sacrum • Dislocated total hip replacement | |
| Major C: Potentially fatal (major head injury [including severe traumatic brain injury such as subdural hematoma, intracranial hemorrhage, subarachnoid hemorrhage]; and hip fracture [including femoral neck and proximal femur fractures]) | • Basilar skull fracture • Acute epidural hematoma • Displaced or nondisplaced proximal femur fracture |
Of 217 Moderate and Major injuries categorized following the IFCAM, our team determined that 31.34% (n = 68) were Moderate and 68.66% (n = 149) were Major injuries. Of the Major injuries, the distribution of Major A, B, and C was 40.27% (n = 60), 16.11% (n = 24), and 43.62% (n = 65), respectively.
Using the injurious fall cases from 2016 to 2018 (n = 232) from 12 participating hospitals, 12 investigators achieved a weighted kappa .96 (.95–.97) and a probability of pair agreement of .92 (.88–.95).
Discussion
We successfully refined the Major injury classification to derive a reliable assessment and categorization of the types and severities of Major inpatient fall-related injuries. Using the IFCAM, trained personnel can rate injury severity with excellent reliability. The large data set of real-world incident reports provides insight on how to reduce limitations identified by Garrard’s team to strengthen the existing NDNQI (Table 3).
Limitations of the NDNQI as Identified by Garrard and Strategies to Overcome
| Limitation | Evidence of Limitation | Strategy to Address Limitation |
|---|---|---|
| Confusion over NDNQI definitions (21) | In the Garrard article, 70–90% of respondents were able to accurately classify the injury levels for 12 of 15 cases, but there was a very poor agreement for three “tricky cases” (21). The 95% confidence intervals for the proportion of cases that were correctly rated ranged from (0.00 to 0.68) to (94.53 to 98.08) with confidence intervals in the poor to moderate range for over half of the cases (21). The authors concluded that the respondents’ ability to correctly classify Minor injuries did not imply that they could also correctly classify Moderate or Major injuries or vice versa (21). In our study, confusion around how to interpret Moderate and Major injuries was apparent in the misclassification of fractures not requiring casting, traction, or surgery (eg, rib fracture, pelvic fracture). | We removed treatment from the definition. We added specific examples of injuries from falls into the Major subcategory operational definitions and the IFCAM. This addressed Garrard and colleagues’ recommendation for more training and education to overcome the confusion of definitions. |
| Ambiguity of fall scenarios (21) | Incomplete and low-quality incident reports are common and can result in erroneous classification if proper follow-up is neglected or not documented. We identified missing or unclear information in 40% of incident reports. Since imaging results are often not reflected in the incident report, this issue seems to have to do with the timing of incident report completion. | By including all data associated with the hospitalization in the incident report and using our administrative manual to classify the 217 real-world Moderate and Major fall injuries included in this study, the overall agreement was 96.5% and the 95% confidence interval was 94.20 to 98.12, indicating excellent reliability. We propose increasing the amount of time for completing the severity rating from 24 hours (as currently indicated by the NDNQI) (21) to the end of a patient’s length of stay to allow for more complete documentation. Given our team is experienced with rating falls across three large hospital systems, we determined that the final severity rating should be decided by trained personnel who are equipped with a valid administration manual, the IFCAM (Supplementary Appendix B) and have access to complete information including relevant follow-up from a patient’s entire inpatient stay. This will result in accurate incident reports and severity ratings. |
| Bias from evaluator’s perspectives (21) | Subjectivity of the decision to treat the same injury (eg, wrist fracture) with a splint or a cast was apparent in the overlap we saw between cases rated as Moderate using the NDNQI and our Major subcategories. | By creating subcategories not reliant on the treatment of the injury, we can avoid classification of injuries based on a subjective perception of appropriate treatments. “Tricky cases” were incorporated into the operational definitions and the IFCAM to improve interrater reliability. |
| Limitation | Evidence of Limitation | Strategy to Address Limitation |
|---|---|---|
| Confusion over NDNQI definitions (21) | In the Garrard article, 70–90% of respondents were able to accurately classify the injury levels for 12 of 15 cases, but there was a very poor agreement for three “tricky cases” (21). The 95% confidence intervals for the proportion of cases that were correctly rated ranged from (0.00 to 0.68) to (94.53 to 98.08) with confidence intervals in the poor to moderate range for over half of the cases (21). The authors concluded that the respondents’ ability to correctly classify Minor injuries did not imply that they could also correctly classify Moderate or Major injuries or vice versa (21). In our study, confusion around how to interpret Moderate and Major injuries was apparent in the misclassification of fractures not requiring casting, traction, or surgery (eg, rib fracture, pelvic fracture). | We removed treatment from the definition. We added specific examples of injuries from falls into the Major subcategory operational definitions and the IFCAM. This addressed Garrard and colleagues’ recommendation for more training and education to overcome the confusion of definitions. |
| Ambiguity of fall scenarios (21) | Incomplete and low-quality incident reports are common and can result in erroneous classification if proper follow-up is neglected or not documented. We identified missing or unclear information in 40% of incident reports. Since imaging results are often not reflected in the incident report, this issue seems to have to do with the timing of incident report completion. | By including all data associated with the hospitalization in the incident report and using our administrative manual to classify the 217 real-world Moderate and Major fall injuries included in this study, the overall agreement was 96.5% and the 95% confidence interval was 94.20 to 98.12, indicating excellent reliability. We propose increasing the amount of time for completing the severity rating from 24 hours (as currently indicated by the NDNQI) (21) to the end of a patient’s length of stay to allow for more complete documentation. Given our team is experienced with rating falls across three large hospital systems, we determined that the final severity rating should be decided by trained personnel who are equipped with a valid administration manual, the IFCAM (Supplementary Appendix B) and have access to complete information including relevant follow-up from a patient’s entire inpatient stay. This will result in accurate incident reports and severity ratings. |
| Bias from evaluator’s perspectives (21) | Subjectivity of the decision to treat the same injury (eg, wrist fracture) with a splint or a cast was apparent in the overlap we saw between cases rated as Moderate using the NDNQI and our Major subcategories. | By creating subcategories not reliant on the treatment of the injury, we can avoid classification of injuries based on a subjective perception of appropriate treatments. “Tricky cases” were incorporated into the operational definitions and the IFCAM to improve interrater reliability. |
Limitations of the NDNQI as Identified by Garrard and Strategies to Overcome
| Limitation | Evidence of Limitation | Strategy to Address Limitation |
|---|---|---|
| Confusion over NDNQI definitions (21) | In the Garrard article, 70–90% of respondents were able to accurately classify the injury levels for 12 of 15 cases, but there was a very poor agreement for three “tricky cases” (21). The 95% confidence intervals for the proportion of cases that were correctly rated ranged from (0.00 to 0.68) to (94.53 to 98.08) with confidence intervals in the poor to moderate range for over half of the cases (21). The authors concluded that the respondents’ ability to correctly classify Minor injuries did not imply that they could also correctly classify Moderate or Major injuries or vice versa (21). In our study, confusion around how to interpret Moderate and Major injuries was apparent in the misclassification of fractures not requiring casting, traction, or surgery (eg, rib fracture, pelvic fracture). | We removed treatment from the definition. We added specific examples of injuries from falls into the Major subcategory operational definitions and the IFCAM. This addressed Garrard and colleagues’ recommendation for more training and education to overcome the confusion of definitions. |
| Ambiguity of fall scenarios (21) | Incomplete and low-quality incident reports are common and can result in erroneous classification if proper follow-up is neglected or not documented. We identified missing or unclear information in 40% of incident reports. Since imaging results are often not reflected in the incident report, this issue seems to have to do with the timing of incident report completion. | By including all data associated with the hospitalization in the incident report and using our administrative manual to classify the 217 real-world Moderate and Major fall injuries included in this study, the overall agreement was 96.5% and the 95% confidence interval was 94.20 to 98.12, indicating excellent reliability. We propose increasing the amount of time for completing the severity rating from 24 hours (as currently indicated by the NDNQI) (21) to the end of a patient’s length of stay to allow for more complete documentation. Given our team is experienced with rating falls across three large hospital systems, we determined that the final severity rating should be decided by trained personnel who are equipped with a valid administration manual, the IFCAM (Supplementary Appendix B) and have access to complete information including relevant follow-up from a patient’s entire inpatient stay. This will result in accurate incident reports and severity ratings. |
| Bias from evaluator’s perspectives (21) | Subjectivity of the decision to treat the same injury (eg, wrist fracture) with a splint or a cast was apparent in the overlap we saw between cases rated as Moderate using the NDNQI and our Major subcategories. | By creating subcategories not reliant on the treatment of the injury, we can avoid classification of injuries based on a subjective perception of appropriate treatments. “Tricky cases” were incorporated into the operational definitions and the IFCAM to improve interrater reliability. |
| Limitation | Evidence of Limitation | Strategy to Address Limitation |
|---|---|---|
| Confusion over NDNQI definitions (21) | In the Garrard article, 70–90% of respondents were able to accurately classify the injury levels for 12 of 15 cases, but there was a very poor agreement for three “tricky cases” (21). The 95% confidence intervals for the proportion of cases that were correctly rated ranged from (0.00 to 0.68) to (94.53 to 98.08) with confidence intervals in the poor to moderate range for over half of the cases (21). The authors concluded that the respondents’ ability to correctly classify Minor injuries did not imply that they could also correctly classify Moderate or Major injuries or vice versa (21). In our study, confusion around how to interpret Moderate and Major injuries was apparent in the misclassification of fractures not requiring casting, traction, or surgery (eg, rib fracture, pelvic fracture). | We removed treatment from the definition. We added specific examples of injuries from falls into the Major subcategory operational definitions and the IFCAM. This addressed Garrard and colleagues’ recommendation for more training and education to overcome the confusion of definitions. |
| Ambiguity of fall scenarios (21) | Incomplete and low-quality incident reports are common and can result in erroneous classification if proper follow-up is neglected or not documented. We identified missing or unclear information in 40% of incident reports. Since imaging results are often not reflected in the incident report, this issue seems to have to do with the timing of incident report completion. | By including all data associated with the hospitalization in the incident report and using our administrative manual to classify the 217 real-world Moderate and Major fall injuries included in this study, the overall agreement was 96.5% and the 95% confidence interval was 94.20 to 98.12, indicating excellent reliability. We propose increasing the amount of time for completing the severity rating from 24 hours (as currently indicated by the NDNQI) (21) to the end of a patient’s length of stay to allow for more complete documentation. Given our team is experienced with rating falls across three large hospital systems, we determined that the final severity rating should be decided by trained personnel who are equipped with a valid administration manual, the IFCAM (Supplementary Appendix B) and have access to complete information including relevant follow-up from a patient’s entire inpatient stay. This will result in accurate incident reports and severity ratings. |
| Bias from evaluator’s perspectives (21) | Subjectivity of the decision to treat the same injury (eg, wrist fracture) with a splint or a cast was apparent in the overlap we saw between cases rated as Moderate using the NDNQI and our Major subcategories. | By creating subcategories not reliant on the treatment of the injury, we can avoid classification of injuries based on a subjective perception of appropriate treatments. “Tricky cases” were incorporated into the operational definitions and the IFCAM to improve interrater reliability. |
Overlap exists between the injuries our team classified as Major and the apparent real-world perception of the NDNQI’s Moderate injury level, which is partially based on treatment (21). Of 45 injurious inpatient falls between 2010 and 2015 originally rated as Moderate, 20 (42%) would have been considered either Major A, B, or C by our operation definitions. And of 65 injuries from 2016 to 2018, 16 (25%) rated Moderate would have been considered either Major A, B, or C by our operational definitions. The reason this number is much lower in the 2016–2018 cases is likely due to the removal of 17 cases rated Moderate that had insufficient details to confirm the level of injury. Most overlap involved fractures where no surgery, traction, or casting was required (eg, vertebral fracture, pelvic fracture, rib fracture). There is ambiguity in the NDNQI as to whether a broken bone treated with a sling should be classified as Moderate (including treatment with a sling) or Major (including internal injury) (21). Based on the incident reports in our study, many nurses based their classifications solely on treatment in practice. One of the cases rated Moderate (with the NDNQI) involved a proximal tibia fracture where the patient refused surgery and instead simply had the joint immobilized. Had this patient opted for surgery, the rater would have likely elevated this same injury to the Major classification. In our Major subcategories, this proximal tibia fracture would have been rated as Major B due to its association with an increased Standardized Mortality Ratio and potential for significant long-term morbidity (30,31). To address such misinterpretation of the Major subcategories, we provide an abundance of real case examples in our operational definitions and the IFCAM.
Four head injury cases in the 2016–2018 incident reports were classified as Moderate by our team, while they were previously rated as Major with the NDNQI ratings. All these cases were superficial injuries or hematomas with no abnormal clinical findings from imaging. Clarification as to whether these were initially misclassified in the hospital setting or whether there is true overlap in this area is required. By expanding the definitions of Major injuries from falls, and providing clear examples to raters, we can improve reliability by fleshing out where injuries should be classified based on their functional and mortality implications.
Limitations
Major subcategories are based on the literature available at the time. New studies may emerge requiring reconsideration of certain injury placement within the three subcategories.
The authors recognize that outcomes may change after hospitalization and may not be fully reflected in the incident report or medical record. Assessments of injury severities rely on the accuracy and completion of incident reports and supplementary information collected from the EHR throughout the patients’ length of stay.
While the Major injury subcategories are independent of treatment, we recognize that treatment can be an indicator of severity and can also affect the outcome; for example, surgery can increase the risk of mortality. Factors including age, frailty, level of assistance, socioeconomic factors, comorbidities, environmental barriers, and quality of health care affect the level of burden patients experience from their injuries (32). It was not feasible to consider all those individual factors that can influence injury severity for specifying subcategory operational definitions applicable to the general inpatient population.
Next Steps
This investigation has primarily focused on Major injuries from falls. Additional research should be conducted on each of the remaining NDNQI injury severity categories to improve reliability. For example, for a patient who fell and hit their knee, if the nurse does nothing, the injury severity is rated as “None.” However, if the nurse applies ice, the severity is rated as “Minor.” As mentioned earlier, per NDNQI, Moderate injuries require suturing, application of steri-strips/skin glue, and splinting of muscle/joint strain. For rating Moderate injuries in this study, we expanded the NDNQI definition to include “injuries that are expected to heal quickly and leave no residual impairment,” and which had “no impact on hospitalization because of fall.” Additional research is required to remove injury treatment, which can be subjective, from all NDNQI injury definitions.
Conclusions
Accurate assessment of the injury severity and impact from inpatient falls informs the development of appropriate education, risk assessment, and fall prevention plans (21). The reliability of injury severity ratings is dependent on both the completeness and accuracy of data recorded in incident reports and when necessary, from the patient’s medical record and the quality and objectivity of the rater. The IFCAM can be used to improve the consistency of inpatient injurious fall severity ratings for hospitals using the NDNQI and for those searching for a standard way of quantifying major harm from falls. For hospitals using the NDNQI, we suggest two modifications to improve current practice. Our first recommendation is for hospitals to use the IFCAM to identify and further classify Major injuries from falls based on the mutually exclusive, objective categories: Major A, B, and C. The second recommendation is for hospitals to identify existing infrastructure that would allow for routine review of injurious fall severity classifications not just 24 hours after the incident (21), but to document changes in injury status for the patient’s entire length of stay.
Acknowledgments
The authors would like to thank the clinicians at Partners HealthCare System, Montefiore Medical Center, and New York-Presbyterian who participated in the creation and validation of the Major subcategories. The authors would also like to thank the medical librarian at Countway Library of Medicine (an alliance of the Harvard Medical School and Boston Medical Library) for their assistance with the systematic literature review. All authors contributed significantly to this paper.
Funding
This work was supported by the Agency for Healthcare Research and Quality (grant number 1R18HS025128-01).
Conflict of Interest
The authors have no conflicts of interest to report.
