Successful Healthcare Improvements From Translating Evidence in complex systems (SHIFT-Evidence): simple rules to guide practice and research

Abstract

Background

Evidence translation and improvement research indicate that healthcare contexts are complex systems, characterized by uncertainty and surprise, which often defy orchestrated intervention attempts. This article reflects on the implications of complexity on attempts to translate evidence, and on a newly published framework for Successful Healthcare Improvements From Translating Evidence in complex systems (SHIFT-Evidence).

Discussion

SHIFT-Evidence positions the challenge of evidence translation within the complex and evolving context of healthcare, and recognizes the wider issues practitioners routinely face. It is empirically grounded, and designed to be comprehensive, practically relevant and actionable. SHIFT-evidence is summarized by three principles designed to be intuitive and memorable: ‘act scientifically and pragmatically’; ‘embrace complexity’; and ‘engage and empower’. Common challenges and strategies to overcome them are summarized in 12 ‘simple rules’ that provide actionable guidance.

Conclusion

SHIFT-Evidence provides a practical tool to guide practice and research of evidence translation and improvement within complex dynamic healthcare settings. Implications are that improvement initiatives and research study designs need to take into account the unique initial conditions in each local setting; conduct needs to respond to unpredictable effects and address dependent problems; and evaluation needs to be sensitive to evolving priorities and the emergent range of activities required to achieve improvement.

complexity, complex adaptive systems, quality improvement, evidence translation, implementation, framework

Introduction

The four World Health Organisation resolutions (availability, accessibility, acceptability and quality of services) reflect a global ambition to ensure people receive effective, evidence-based treatments in care, that meet the needs and expectations of service users, and optimize use of resources invested in healthcare systems [1–3]. Whilst examples of successful improvement exist, achieving reliable and consistent improvements to care quality remains a major international challenge [4–7]. Further insights are required to inform how improvement initiatives are designed and conducted to optimize chances of success, and to inform how evaluations can be conducted to generate knowledge that support reproduction of improvements in other settings [8–11].

Research into the implementation of evidence-based practices and quality improvement efforts highlight context and complexity as two major challenges. The success of an intervention tends to be dependent on the particular context (the place, setting or environment) into which a change is implemented [12]. Interventions and implementation strategies need to be adapted to fit with local cultures, practices and systems of care to increase chances of success, making it difficult to know in advance what will work, for whom, in what settings [13–15].

It is proposed that healthcare contexts should be considered as complex and dynamic systems, with associated characteristics of agency, interconnectedness, dynamism and unpredictability [16–18]. The agency of people in a system means that it can be difficult to predict how they will react to new interventions. Healthcare professionals/staff are autonomous and highly skilled which will influence their willingness to engage with (or disrupt) proposed changes, and affect how they interpret and adapt changes to meet their needs. System interconnectedness means that people, behaviours and processes all interact. This makes it challenging to focus on any individual component of a system, or to consider an intervention in isolation from the context in which it will be implemented. The dynamic nature of complex systems means that they continually evolve. This makes each setting somewhat unique, influenced by how previous activities and events have affected individuals and the way they work together. These unique initial conditions influence what happens in the future, creating unpredictability and surprise. This means complex systems can defy orchestrated attempts to intervene in them [19, 20]: on one hand, seemingly obvious solutions can fail to deliver their intended benefits, and on other the other, apparently minor changes can have major consequences. Emergent patterns, structures and routines define the system, and guide behaviour within it [18, 21]. Complexity science proposes that this emergent behaviour can be understood through the identification of ‘simple rules’ [19, 22].

Whilst the complexity of healthcare systems is increasingly recognized [21, 23, 24], the full implications of complexity on the design, conduct and evaluation of evidence translation and improvement attempts have not yet been described [25]. There is a need for a greater clarity about how to translate evidence and achieve improvements in complex systems, and how to study them [21, 26].

This article considers the implications of system complexity on attempts to translate evidence and make improvements, and considers a new framework which is designed to provide practical guidance on how to understand and influence complex systems, through a series of ‘simple rules’ that are accessible to practitioners and useful to researchers and evaluators [27]. This article is divided into three sections. The new conceptual framework Successful Healthcare Improvements From Translating Evidence into practice (SHIFT-Evidence) is presented first, followed by an example project narrative that exemplifies how system complexity was experienced in practice and how the ‘simple rules’ supported project success. Finally, we reflect on the implications of this framework for the design, conduct and evaluation of improvement.

Successful Healthcare Improvements From Translation of Evidence into practice

The theory of SHIFT-Evidence can be summarized as: ‘to achieve successful improvements from evidence translation in healthcare it is necessary to act scientifically and pragmatically whilst embracing the complexity of the setting in which change takes place and engaging and empowering those responsible for and affected by the change.’

SHIFT-Evidence is summarized by three strategic principles; act scientifically and pragmatically; embrace complexity; and engage and empower; and 12 associated ‘simple rules’ (see Table 1).

Table 1

Summary of the SHIFT-Evidence strategic principles, common challenges and simple rules

Strategic principle	Common challenges	Simple rules
Act scientifically and pragmatically: Knowledge of existing evidence needs to be combined with knowledge of the unique initial conditions of a system. Interventions need to adapt as the complex system responds and learning emerges about unpredictable effects.	Pre-selected interventions may not solve the problems of the local system	Understand the problem and opportunities
	‘Evidence’ and interventions need to be perceived as locally relevant and actionable	Identify, test and iteratively develop potential solutions
	Individual perceptions of system performance are unreliable	Assess whether improvement is achieved, capture and share learning
	Interventions need to be reviewed and adapted as systems evolve overtime	Invest in continual improvement
Embrace complexity Evidence-based interventions only work if related practices and processes of care within the complex system are functional. Evidence-translation efforts need to identify and address any problems with usual care, recognizing this typically includes a range of interdependent parts of the system.	Interventions don’t work on their own—they need to fit with practices and processes of care	Understand practices and processes of care
	There is rarely a single, standardized, way by which care is delivered	Understand types and sources of variation
	It cannot be assumed that dependent processes or systems are working well	Identify systemic issues
	Any intervention will compete for attention and resource with other initiatives or requirements	Seek political, strategic and financial alignment
Engage and empower Evidence translation and system navigation requires commitment and insights from staff and patients with experience of the local system. Changes need to align with their motivations and concerns.	If people are not motivated change will not take place, and without their engagement, insights will be lost	Actively engage those responsible for and affected by change
	Expect conflict and tension	Facilitate dialogue
	Underlying expectations are to get it right, first time and quickly	Build a culture of willingness to learn and freedom to act
	Improving complex systems takes time, effort and reflection	Provide headroom, resources, training and support

The evidence translation and improvement process described in SHIFT-Evidence is progressive and iterative. The principle ‘act scientifically and pragmatically’ represents the high level stages of an improvement initiative, acknowledging that moving through these stages is unlikely to be a smooth predictable process. The principles ‘embrace complexity’ and ‘engage and empower’ need to be considered throughout each stage of an improvement initiative. These three principles reflect the constant iterative process of navigating systems and responding to emergent findings, and negotiating changes with people responsible for and affected by change. The ‘simple rules’ seek to provide guidance for practice and research in complex systems, allowing stakeholders to understand and respond to emerging challenges and capture learning. This is represented schematically in Figure 1.

Figure 1

The SHIFT-Evidence framework represented schematically, including the three strategic principles (act scientifically and pragmatically, embrace complexity, and engage and empower) and the 12 ‘simple rules’, and demonstrating the continual learning process required for evidence translation and improvement in healthcare settings. Source: [27].

The principles and simple rules of the SHIFT-Evidence framework reflect the implications of intervening in complex systems. The way complex systems are influenced by past events and evolve over time means each system is unique and will respond uniquely, so anticipating all the changes that will be required in an individual setting is therefore not possible. Local knowledge needs to be combined with scientific knowledge from evidence-based medicine to understand the local situation and identify suitable potential solutions. Pragmatic adaptation of interventions is required to embed and sustain changes to practice in each new system.

The interconnectedness of system components means that the implementation of evidence-based interventions is achieved within the local context and care system. Multiple interventions are likely to be required to address the many interconnected needs of the system to achieve any specific improvement goal. This complexity needs to be embraced; attempts to focus solely on isolated activities, individual people or individual ‘solutions’ are unlikely to be successful or achieve maximum impact.

Staff and patients within local organisations hold the majority of relevant knowledge about local systems and the agency and authority to achieve successful changes. Local stakeholders must be engaged and empowered to ensure they are willing and able to participate in translation and improvement efforts, and understand how to effect change in complex systems.

Medicines management: an example project narrative

To demonstrate the practical reality of intervening in complex systems, we present an example narrative from a project conducted by the National Institute for Health Research (NIHR) Collaboration for Leadership in Applied Health Research and Care Northwest London (CLAHRC NWL) [28]. The narrative was selected from the 22 evidence-translation projects reviewed in this research. This medicines management project narrative illustrates the challenges the project team experienced in implementing an evidence-based intervention, and how the ‘simple rules’ were applied to support project success. The challenges it presents were typical. It demonstrates the dependency of the evidence-based intervention on the implementation context and the wider system issues that needed to be addressed.

The project aimed to implement an evidence-based post-discharge follow-up phone call to support patients whose medications had been changed during an emergency admission [29–33]. A high level description of the challenges the team faced in implementing this intervention and the resulting work that was required to improve patient care is summarized in Figure 2.

Figure 2

A high level overview of the process, challenges and actions of the project team to implement a new medicines management process in a complex system.

The medicines management project narrative highlights that before the project team could proceed it was necessary to ‘understand problems and opportunities’ in the local setting; dependent processes (medicines reconciliation at discharge from hospital) needed to be improved before the planned intervention (a follow up phone call) could be implemented. Reliable implementation of evidence required the project team to ‘understand practices and processes for care’; assumptions cannot be made about the reliability or coordination of individual processes and practices. They needed to develop an understanding of ‘work as is’ rather than ‘work as imagined’ to identify and improve related issues that influenced their improvement goal. This in-depth understanding of the local system could only be achieved by ‘actively engaging people responsible for and affected by proposed changes’ (healthcare professionals and patients).

Once the project team understood the changes required in the local system (to improve the accuracy of medications in discharge summaries, and to improve coordination and communication between the four professional groups involved) they could then move on to ‘identify, test and iteratively develop potential solutions’. Developing a common medicines form required the project team to carefully ‘facilitate dialogue’ between the four professional groups to understand what was feasible and acceptable, to ensure it fit with established local practices, and to negotiate perceived threats to roles and responsibilities of the different groups involved. Even after the form was developed the project team continued to ‘identify systemic issues’ that were affecting success of the improvement initiative: phone calls depended on accurate information, accurate information depended on medicines reconciliation, medicines reconciliation depended on staff availability, competencies and joined-up procedures. Achieving the original improvement required many other aspects of the system to be ‘fixed’. The evidence therefore acted as a catalyst for a more comprehensive, complex and challenging system-wide analysis and an improvement process that required support and action from the wider organisation.

The team worked in a ‘culture of willingness to learn and freedom to act’ which encouraged them to recognize and respond to these wider system issues. The project team were given permission to move beyond the initial project scope (implementing a phone call) to address other issues, and supported to seek wider collaborations and engagement when issues were out of their direct sphere of influence (including education and system leaders). This learning culture was supported by the use of objective measurement to ‘assess whether improvement had been achieved’ (the error rate in medicines reconciliation at discharge). This data was presented to stakeholders to build motivation for change, and used by the project team to ‘understand variations’ in how care was provided which in turn informed the selection and iterative development of additional interventions (educational sessions for junior doctors and changes to pharmacy staff rotas).

This process of enquiry, driven by the iterative use of data and feedback from healthcare professionals and patients, led the project team to ‘invest in continuous improvement’, including awareness of issues that threatened the sustainability of the changes they had put in place (including junior doctor rotation, and other emerging organisational priorities). The project team made this possible by seeking ‘political, strategic and financial alignment’: at the start of the project medicines reconciliation had poor visibility within the hospital and was not an organisational priority. The team worked to increase its profile, identifying how the work related to key hospital concerns including the importance of medicines reconciliation to admissions avoidance, how it linked to the safe and effective flow of patients through emergency care, and how it contributed cost-savings by avoiding inappropriate prescribing. The project team also benefited from the ‘provision of headroom, resource, training and support’ from the CLAHRC NWL programme who invested in resourcing dedicated time for core project team members and provided training to empower them to navigate and negotiate change in complex systems.

Implications

Successful Healthcare Improvement From Translating Evidence in complex systems (SHIFT-Evidence) provides, in 3 strategic principles and 12 ‘simple rules’, a comprehensive summary of how to understand and intervene in complex systems for successful evidence translation and improvement.

The ‘simple rules’ aim to make complexity navigable (whilst recognizing that it will never be simple), providing actionable guidance to both practice and research. Our perspective is that all SHIFT-Evidence strategic principles and ‘simple rules’ are necessary to achieve successful sustained improvements in care. Neglecting a single principle may reduce the likelihood of long-term success.

The SHIFT-Evidence framework builds on existing work that has described healthcare as a complex system [16], and conceptualized interventions as events in systems [34, 35]. It addresses a known gap in the literature, considering the implications of complexity for deliberate attempts to intervene and introduce evidence-based practices [21, 25, 26]. The SHIFT-Evidence conceptual framework and 12 ‘simple rules’ provides practitioners (and policy makers), with a synthesis of literature and multiple empirical studies can provide practical guidance to translate evidence in healthcare settings; and for researchers, a ‘real-world’ conceptualization of the challenge of evidence translation and the work required to achieve improvements in care can inform the design and conduct of studies aiming to have high usability, applicability and relevance. We recognize the importance of assessing how this new framework compares with existing frameworks and how it is situated within the growing library of published frameworks and models for implementation and improvement research. As such, we have conducted an exploratory comparative framework analysis which is published in a parallel paper in this journal [36].

Our experience of working at the interface between academia and practice suggests the framework has the potential to support collaboration between practitioners and researchers by providing a common language and conceptualization.

Intervening to achieve improvement

SHIFT-Evidence can help everyone interested in effecting change in healthcare systems (including healthcare practitioners, managers, patients, policy makers, management consultants, industry partners and academics) anticipate the implications of intervening in complex systems. As each system is somewhat unique, a solution may not work in all settings. Implementing evidence-based practices to achieve improvements in care may require multiple, multifaceted interventions responding to emergent learning and adapted to work in the specific setting of use. As such, it might be beneficial if terminology shifts from the noun ‘intervention’, which implies a closed or defined process, to the verb ‘intervening’ as ‘intervening to achieve an improvement’ might better reflect the negotiated and iterative processes necessary to understand and influence complex systems. These insights have implications for the design, conduct and evaluation of improvement initiatives.

Working with the uniqueness of each system

Time must be invested at the design stage to explore problems and opportunities, clarify the improvement goal and prepare to learn. System complexity means there may be different problems or opportunities in each setting, and time and resource is required to investigate this and to develop a shared goal [37]. Interventions (including evidence-based practices) need to be aligned with local problems and opportunities (including understanding what other improvement work is taking (or has taken) place), and need to fit with what matters to practitioners, patients and wider organisational or system priorities [38–40]. As challenges emerge, learning (and evaluation) are key to inform understanding of the system and the iterative development of change ideas [41]. Learning and evaluation should therefore be considered in the design phase in partnership with local stakeholders. The design phase should anticipate that improvement is not about one-off interventions, but building local capacity and capability to continuously adapt to dynamic and evolving contexts [42–44].

Make learning (and re-learning) a priority

During the conduct of an improvement initiative, it is necessary to invest time to understand the influence of interventions on the system in real-time, understand system interconnectivity and respond to emergent learning [45]. Working in complex systems means new learning is likely to challenge initial expectations or assumptions, and a new (revised) set of planned next steps will be regularly required [46]. Such learning includes: understanding how interventions have been received by practitioners and patients; whether tested interventions have had their desired impact or need to be revised; and identifying the dependent processes and practices that need to be in place to support the uptake and impact of a particular intervention. Practically, this requires; time for frequent discussions supported by regular feedback and progress monitoring; permission to revise plans, with setbacks embraced as learning rather than failure and caution when making decisions about where to invest efforts for the greatest gain, balancing the need to address systemic issues with the risk of trying to tackle intractable problems [47, 48]. Iteration and learning can be informed by previous improvement attempts, drawing on peer-to-peer learning, and academic and grey literature.

Unpacking the black box

Evaluations of improvement initiatives need to assess whether the improvement goal has been achieved, understand how the change was achieved, and consider what learning is useful to share with others. To be of value, evaluations need to be prepared to be flexible and adaptable and to adjust focus as required in response to local learning (as opposed to fixing on a predefined outcome and intervention to be studied). Rigorous measurement of an improvement goal, and appropriate analysis, is critical to ascertain whether a change is an improvement [49, 50]. Regular review of process and/or outcome measures should provide formative feedback to inform learning and guide decision making. When working in complex systems, there is less value in a reductionist approach: trying to attribute simple causal relationships or isolate the effect of an intervention independent of its context. Evaluations should endeavour to unpack the black box of improvement [51], understanding the way in which interventions interact with and influence the setting they are deployed in, what adaptations were made to make the intervention usable and effective locally, and the dependent problems that were identified and how they were resolved [52, 53]. Evaluations and knowledge outputs should be designed to maximize their usefulness to both inform progress in the local setting, and to share learning with people seeking to reproduce improvements elsewhere, helping them to anticipate the work that is required and the issues they might encounter in their local setting.

Applying SHIFT-Evidence in practice

The next step is for people to use SHIFT-Evidence in practice, research or policy-making and assess the practical utility of the SHIFT-Evidence framework, and its value in guiding improvement initiatives and research studies. Whilst the ‘simple rules’ provide high level guidance we anticipate that much more work will be required to understand how to effectively operationalize these in practice.

Conclusion

SHIFT-Evidence and its ‘simple rules’ provides an empirically grounded, theoretical framework which is also capable of acting as a practical tool to understand and guide evidence translation and improvement within complex dynamic healthcare systems.

The principles and simple rules of the SHIFT-Evidence framework have implications for the practice and research of evidence translation and improvement. Change initiatives and their research study designs need to take into account the unique initial conditions in each local setting, the way they are conducted needs to respond to unpredictable effects and address dependent problems and evaluation needs to be sensitive to evolving priorities and the emergent range of activities required to achieve improvement.

SHIFT-Evidence has the potential to provide a common platform for academics, practitioners, patients and policymakers to work collaboratively to achieve improvements in healthcare.

Supplementary material

Supplementary material is available at International Journal for Quality in Health Care online.

Acknowledgements

Authors would like to thank Dr Catherine French, Stuart Green, Rachel Matthews, Laura Lennox, Ganesh Sathyamoorthy and Dr Tom Woodcock for their advice and support in development of the framework and review of the text; and Dr Louella Vaughn and Dr Vanessa Marvin who led the Medicines Management project and reviewed and commented on the project narrative used in this paper. This paper would not have been possible without the hard work of these colleagues, and all the CLAHRC NWL community.

Funding

This article presents independent research supported by the National Institute for Health Research (NIHR) under the Collaborations for Leadership in Applied Health Research and Care (CLAHRC) programme for North West London. JR was also financially supported by an Improvement Science Fellowship from the Health Foundation. The views expressed in this publication are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health and Social Care or The Health Foundation.