Improvement capability and performance: a qualitative study of maternity services providers in the UK

Abstract Objective We explore variations in service performance and quality improvement across healthcare organisations using the concept of improvement capability. We draw upon a theoretically informed framework comprising eight dimensions of improvement capability, firstly to describe and compare quality improvement within healthcare organisations and, secondly to investigate the interactions between organisational performance and improvement capability. Design A multiple qualitative case study using semi-structured interviews guided by the improvement capability framework. Setting Five National Health Service maternity services sites across the UK. We focused on maternity services due to high levels of variation in quality and the availability of performance metrics which enabled us to select organisations from across the performance spectrum. Participants About 52 hospital staff members across the five case studies in positions relevant to the research questions, including midwives, obstetricians and clinical managers/leaders. Main Outcome Measure A qualitative analysis of narratives of quality improvement and performance in the five case studies, using the improvement capability framework as an analytic device to compare and contrast cases. Results The improvement capability framework has utility in analysing quality improvement within and across organisations. Qualitative differences in the configurations of improvement capability were identified across all providers but were particularly striking between higher and lower performing organisations. Conclusions The improvement capability framework is a useful tool for healthcare organisations to assess, manage and develop their own improvement capabilities. We identified an interaction between performance and improvement capability; higher performing organisations appeared to have more developed improvement capabilities, though the meaning of this relationship requires further research.


Introduction
The highly variable progress and impact of quality improvement programmes in healthcare organisations [1] is often attributed to two factors: the improvement intervention itself and the fidelity with which it is implemented [2]; and the nature, history, trajectory and other attributes of the organisation in which it is implemented, often labelled organisational context [3,4]. There have been a number of approaches and attempts to provide guidance on how to describe improvement interventions and their evaluation [5] to describe the salient features of organisational context [6], and to advocate for research and implementation to take greater account of such contextual factors [7]. The division of intervention and context is arguably somewhat artificial because the two interact in multiple, complex and dynamic ways [8]. Descriptions of contextual factors or attributes without a rationale founded in relevant organisational theory seem unlikely to produce much explanatory insight [9].
However, there is a substantial relevant literature, in healthcare and other sectors, on what has been termed improvement capability, defined for the purposes of this paper as 'the organisational ability to intentionally and systematically use improvement approaches, methods and practices, to change processes and products/services to generate improved performance' [10] (p. 5). This definition comes from a recent integrative review of improvement capability by Furnival et al. which highlighted the absence of empirically tested and validated frameworks for assessing improvement capability and identified a number of competing definitions and 70 different assessment tools or frameworks with somewhat divergent underlying theoretical models, constructs and measurement items. The review produced a synthesis of those constructs into a framework of eight core dimensions of improvement capability: service-user focus, stakeholder focus, organisational culture, employee commitment, leadership commitment, data and performance, process improvement and learning, and strategy and governance. Importantly, it also located the ideas and concepts of improvement capability in the much wider literature on organisational resources and capabilities and their relationships with organisational performance. This 'dynamic capabilities view' [11][12][13][14] has been widely used and empirically tested [15][16][17][18]; though it is not without its critics who highlight a lack of consistency in definitions [19,20], a tautological tendency [21] and an absence of approaches to assessment [22].
We had two main aims in this study. First, we wanted to test out Furnival et al.'s improvement capability framework for the first time empirically, to see whether it was helpful in describing and explaining the development and progress of quality improvement programmes in healthcare organisations, and to see whether we could define the eight dimensions more fully through empirical study. Second, we wanted to explore the interactions between organisational performance and improvement capability, by seeing whether organisations with differing performance on available metrics had different configurations of improvement capability (by which we mean the qualitative description of their capabilities when mapped to the eight dimensions of the improvement capability framework). We used the improvement capability framework to guide our data generation and analysis, which are outlined in the following section.

Setting and sample
We undertook the study in maternity services in the UK because of the availability of a wide range of quantitative performance metrics, the known existence of variations in quality and safety and the high salience of this service area resulting from past high-profile failures in care [23,24]. In addition, because we do not know how much improvement capability might vary across or within healthcare organisations by specialty or clinical service area, we chose maternity services because they tend to be relatively self-contained and less codependent on other specialties than many others in acute care.
We selected case studies of National Health Service (NHS) maternity service providers in the UK to represent a range of performance on publicly available quantitative metrics drawing on routine data based on hospital admissions, national patient surveys and regulatory reviews [25][26][27][28][29]. We assembled a set of 38 indicators covering four domains of quality: clinical care, mortality, regulatory assessment and patient experience. Monte Carlo simulation techniques [30] were then used to model the results of aggregating the indicators using many different relative weightings and we then calculated how often each provider appeared in the top or bottom decile of the performance distribution.
This method identified eight providers which appeared in the top decile of performance on more than 50% of occasions and eleven which appeared in the bottom decile of performance more than 50% of occasions, with the remaining providers being 'middling' performers. We selected five maternity service providers for our case studies based on this quantitative assessment, contextual factors (geography, neonatal care level, size of unit measured in number of deliveries per annum), and willingness to participate. Our cases are described in Table 1.
As Table 1 outlines, there were a wide range of improvement methods used across the five organisations and all had experienced regulatory inspections though with differing outcomes. The organisations had varying levels of external engagement and some differences in their internal management structures.

Instruments and procedures
We undertook semi-structured interviews across the five case studies with 52 hospital staff members in positions relevant to the research questions (e.g. midwives, obstetricians, quality/audit leads, governance and service directors and clinical managers/leaders). Participants were selected to include a range of perspectives and experiences from different levels in each organisation. All interviews followed the same interview schedule, which aimed to explore how improvement happens and to understand any aspects that help or hinder improvement within each organisation. The interview questions were guided by the eight dimensions of the improvement capability framework, and structured questions enabled comparisons to be made across the different cases [31]. Relevant documents, such as action plans, reports and meeting minutes were also reviewed to enhance the understanding of each case [32]. Ethical approval for the research was granted by the University of Manchester and coordinated research governance approval was given by the Health Research Authority (ref: 184263) and the participating NHS organisations. In this paper the case study sites and research participants are anonymised.
Interviews lasted between 30 min and 1 h. Interviews were recorded with the permission of each participant and transcribed verbatim. To address potential informant bias, triangulation of data was achieved by including a range of participants in the study and through the review of documents.

Data analysis
Analysis involved deductive coding of interview transcripts using the improvement capability framework, which was then expanded from the data inductively. This deductive-inductive cycle not only allowed the data to be interpreted through the improvement capability framework, but also enabled the development and expansion of the framework from the data through various subthemes. Cross case analysis was then conducted to interpret whether improvement capability appeared to be configured differently across the differently performing organisations. The software Nvivo was used as a tool to manage and analyse data and all data generated within the study was stored in Nvivo, including audio files, documents and transcripts. These practices helped to avoid data overload and manage the large amount of data commonly collected in case studies [33]. Furthermore, Nvivo provided a useful place to keep an audit trail to detail and justify data collection [34] and support the sincerity and transparency of the research [35]. This audit trail included, for example, all generated data, notes and reflections following data generation and analysis.

Results
We present the findings from the study below. First, we explore whether the eight dimensions of improvement capability derived from the review are found in our qualitative data, and report on the results of inductive coding within each dimension. Second, we compare and contrast the configurations of improvement capability found in our high and low performing providers (Trust A and Trust E). Third, we examine in detail the configurations of improvement capability in two example dimensions-leadership commitment and process improvement and learning-across all case study sites (space precludes such an analysis in this paper for all eight dimensions).

Identifying and describing the dimensions of improvement capability
The improvement capability dimensions [10] were clearly identifiable in all the cases as is illustrated in Table 2, which lists each dimension, the subthemes identified in our analysis, and a typical example from our interview data. The subthemes that emerged from inductive analysis provide a more detailed and nuanced understanding of each dimension and enabled us to 'follow' rather than 'lead' the data [32]. For example, we found that leadership commitment within an organisation covers a wide area and the subthemes that form this dimension help to highlight particular aspects of leadership that affect on improvement, such as the level of support demonstrated by leaders for improvement and leaders' focus on improvement as perceived by staff.

Differences in improvement capability between high and low performing providers
Qualitative differences in the configurations of improvement capability were identified across the case studies but they were most clearly identified in the two cases at each end of the performance range: Trust E, a 'low' performer, and Trust A, a 'high' performer (see Table 3). Trust A A Foundation Trust with~4500 annual births and an LNU; progressed from being placed in special measures a few years ago to currently holding an 'Outstanding' CQC rating for the maternity services department. The organisation had a high level of engagement with external organisations, such as universities, the CCG, the Allied Health Science Network, NICE and RCOG. A blended approach was taken to improvement methods, which included PDSA and lean methods and aimed to provide a toolkit of improvement as part of their improvement academy.
High (92% of the time in the top decile, never in the bottom decile) Trust B A Foundation Trust with~7750 annual births and a SCU and NICU. The maternity services department currently has a 'Good' CQC rating. The organisation had a high level of engagement with some external organisations, such as universities, the regional strategic clinical network and the Allied Health Science Network, No specific methodologies were identified as being followed for improvement.
Middle (1% of the time in top decile,0% of the time in bottom decile) Trust C A Foundation Trust with~6 000 annual births and a NICU; placed in financial special measures last year and the maternity services department is currently rated as 'Requires Improvement' by CQC. The organisation had few links to external organisations and networks. No specific methodologies were identified as being followed for improvement.
Middle (17% of the time in top decile,1% of the time in bottom decile) Trust D A Foundation Trust with~8 000 annual births across two hospitals and a SCU and LNU; contains two recently merged sites, one of which was a poor performer before the merger. The maternity services department was currently rated by the CQC as 'Good'. Strong networks exist with other maternity services in the area and there are partnerships with external organisations to learn about approaches to quality. There was a planned systematic approach to quality improvement with relevant training opportunities.
Middle (0% of the time in top decile,9% of the time in bottom decile) Trust E A Foundation Trust with~3 000 annual births and an LNU; the maternity services department was investigated for a series of perinatal deaths prior to the most recent rating of 'Requires Improvement' by CQC. External standards and pressure from stakeholders had contributed to quality improvements and the drive for improvement seemed to come externally rather than internally. Bodies such as the CQC and RCOG are seen as being needed to bring about change where there is a perceived lack of Trust level support. There was no systematic approach to quality improvement Low (75% of the time in the bottom decile, never in the top decile) Note: LNU, Local Neonatal Unit; SCU, Special Care Unit; NICU, Neonatal Intensive Care Unit.
As can be seen in Table 3, Trusts A and E are starkly different not only in their performance according to the quality metrics but also in their configurations of improvement capability. For example, whereas improvement activities in Trust E were described by the Clinical Governance and Audit Lead as being 'driven towards pleasing external bodies', in Trust A the Supervisor of Midwives described an active focus on service users as being 'a great driver for any quality improvement'. Similarly, staff in Trust A reported feeling able and 'empowered' to initiate improvement and encouraged not just to highlight problems but to also come up with ideas and solutions. Such encouragement was demonstrated through the annual awards for improvement projects and the development of 'improvement champions' to motivate and guide other staff members.
In Trust E, staff described improvement activities as being led by a limited number of proactive individuals rather than collectively as part of directorate culture. There appeared to be no systematic approach to quality improvement and future goals and priorities were described as being driven by ongoing issues, rather than following any 'planned strategy'. In contrast, future goals and priorities in Trust A were collaboratively planned in advance and structural changes had been made across the Trust with the aim of improvement. In particular, much effort had gone into developing and teaching systematic approaches to quality improvement and staff described feeling empowered with a 'toolkit of improvement' that gave them 'ownership' of improvement activity. In short, the large differences in performance according to available quality metrics between Trust A (in the top decile 92% of the time) and Trust E (in the bottom decile 75% of the time) were matched by equally marked differences in their configurations of improvement capability.

Differences in improvement capability across the range of performance
Although the three 'middling' performing cases were not easily distinguishable in the quantitative analysis of performance metrics, we found there were qualitative differences in improvement capability as this section will outline. Broadly, Trust D appeared to be similar in many ways to Trust A (our high performer) in terms of their configuration of improvement capability, whereas Trusts B and C seemed to share some similarities with Trust E, our low performer. Below, we focus on two of the eight improvement capability dimensions (leadership commitment and process improvement and learning), using the subthemes within these dimensions to explore the similarities and differences between the cases in further detail. All eight dimensions could not be discussed in detail within the scope of this paper, but we believe that the two dimensions presented are reflective of findings from the other dimensions.

Leadership commitment
The improvement capability dimension of leadership commitment concerns the support for improvement demonstrated by organisational leaders, which in our study included both trust and divisional leaders. This section presents how the subthemes within this dimension appeared across the cases (summarised in Table 4) and concentrates on differences regarding the support and focus of leaders for improvement activity, and how this is communicated to staff, including the visibility of senior leaders and existing accountability structures.
In Trusts A and D staff perceived strong support for improvement from organisational and divisional leaders which appeared to have a motivational effect. However, in Trusts B, C and E staff expressed feeling a lack of support and interest from leaders, except when something in the service 'goes wrong' or fails: 'We're not really heard or seen unless it goes wrong. When it goes wrong then I'm upstairs talking to the Chief Executive or the Medical Director or the Director of Nursing -what the hell's going on? -but the rest of the time I don't think they're that interested in us to be honest really.' (Clinical Director, Obstetrics, Trust C) The perceived focus of leaders and their values, vision and priorities also impacted on improvement activities. In Trusts A and D leaders were described by staff as having 'a drive towards quality improvement projects' and a focus on quality rather than finance. In The focus of leaders on improvement was communicated in different ways across the cases and the extent to which staff felt able to contribute to this focus also differed. In Trusts A and D communications from and with leaders were described as multidirectional:  Perceived lack of support for and interest in maternity from trust leaders until an emergency.
Perceived lack of support for and interest in maternity from trust leaders until an emergency.
Perceived lack of support for and interest in maternity from divisional leaders.
Perceived support for and interest in maternity from trust leaders.
Perceived support for and interest in maternity from trust leaders.

Focus
Perceived commitment to and valuing of improvement from some trust leaders.
Perceived lack of commitment to and valuing of improvement from trust leaders. 'I would say that we never had a direct access to anybody above the head of midwifery really… I feel that it wasn't a two way street. So they raised their concerns and gave us their objectives, and they got fed down to us via the head of midwifery and matrons, but it almost felt punitive -if you don't do this, if you don't do that. I feel that we weren't afforded the same voice the other way.' (Labour Ward Manager, Trust E) Improvement appeared to be enabled in Trusts A and D through strong support from senior leaders, which was communicated in various ways with opportunities for staff to contribute. These opportunities were rather more lacking in Trusts B, C and E, where staff often felt disconnected from and unimportant to senior leaders.

Process improvement and learning
The improvement capability dimension of process improvement and learning concerns the availability and use of systematic methods and ongoing processes to enable improvement. This section presents how the subthemes within this dimension appeared across the cases (summarised in Table 5) and concentrates on differences regarding methods used for improvement, how learning around these methods is shared and how improvement activity is supported by training and supervision. Methods and tools drawn upon for improvement varied across trusts, from Trust E where staff described a strong reliance on audits to Trust A where a more flexible project-based approach had been developed: '[M]y aim is to give people a toolkit of improvement where they say, this applies here, I'll go and do this. I don't have a kind of one size fits all approach and also if I'm honest I don't really mind what tool you use, as long as it gives you what you need it to give you.' (Improvement Training Lead, Trust A) Staff in Trust A described a systematic approach to quality improvement, going beyond a reliance on audit, which was described as a way to identify a problem but not prescribing how to solve it. A systematic approach was described as lacking in Trusts E and C, which was viewed as a barrier to improvement activity. Staff in Trust B highlighted the use of methodologies such as the PDSA cycle, but also described a similar lack of a systematic approach to improvement: 'There are processes, but I don't know if they're dictated by the Trust or if they're just what people have learned and what they've seen.' (Midwife, Quality and Audit specialist, Trust B) All trusts except Trust E provided training in improvement methods, however staff in Trusts C and B reported that limited resources, in particular staff time, prevented them from participating in such learning opportunities. Time to put 'training into practice' and opportunities to share learning were valued by staff across the cases: A more 'organic', ongoing 'learning on the job' process was described by staff in Trust E, which made monitoring and assessing staff development difficult. In Trusts A and D multidisciplinary training and development provided opportunities for service improvement, which was supported through supervision and appraisals: Staff supervision and appraisals provide opportunities for development and feedback.

Methods
No systematic or planned approach to quality improvement.
No systematic or planned approach to quality improvement.
No systematic or planned approach to quality improvement.
Systematic and planned approach to quality improvement.
Systematic and planned approach to quality improvement. Improvement was enabled in Trusts A and D through a planned systematic approach, along with training and opportunities to share and put learning into practice.

Discussion
We found the concept of improvement capability and the framework of eight dimensions useful in describing and analysing the organisational capabilities of the five maternity service providers in our case studies, and in seeking more than a superficial description of differences in organisational context and making a more theoretically led analysis. We found the subthemes identified inductively from our qualitative data helpful in moving beyond broad descriptions of improvement capability dimensions like leadership commitment to conceptualise and articulate their meaning in more detail.
We have been cautious about describing particular configurations of improvement capability in scalar terms (such as strong and weak, or good and poor) but it seems clear that Trusts A and D in our research had a more developed or mature improvement capability than trusts B, C and E.
Our study provides some empirical evidence that different configurations of improvement capability are associated with differences in organisational performance, a finding which is consistent with prior research on dynamic capabilities [14,18]. However, the nature of any relationship remains unclear. It seems plausible that organisations with more developed or mature improvement capabilities are more able to improve and sustain performance and therefore more likely to either be high performing or to become high performing. Equally, it is probable that organisations which seem to lack improvement capabilities will not be able, on their own, to bring about performance improvements and may need external support, and there is a risk that when such external support is withdrawn improvements may not be sustained unless their own improvement capabilities have been developed.
We think the main value of the improvement capability framework may be in helping those engaged in quality improvement in healthcare organisations to conceptualise, articulate and self-assess their organisations' improvement capabilities in order to better understand how to increase the progress and impact of improvement programmes. The framework could also be useful to external bodies such as regulators or improvement agencies who need to decide where to target their limited resources, and to make assessments about both the current performance and future likely trajectory of organisations, but whoever uses it, we think its main value lies more in formative and developmental assessments of improvement capability, rather than in summative judgments.
We recognise the limitations of our study, which was undertaken in one clinical service area, in a small number of case studies, at one point in time and only included research participants from within rather than external to each organisation. There are a number of areas for further research. We need a better understanding of what we have termed configurations of improvement capability: how the different dimensions in the framework are connected or interrelated. We know little about how improvement capability varies within organisations (e.g. between clinical service areas or specialties), or how it changes over time, and longitudinal research could explore these issues and help provide a better understanding of the associations between improvement capability and performance. If we think an organisation's stock of improvement capabilities is a dynamic resource that requires ongoing work to replenish and maintain, we would surely want to know more about the ways that organisations can develop improvement capabilities, and the interventions they might use. The development of diagnostic tools for assessing or measuring improvement capability could be pursued, though we are cautious about how quantifiable these essentially qualitative attributes may be, and aware of the risks that attempts at measurement and quantification could create perverse incentives and behaviours [36,37].

Conclusion
This paper has presented an empirical study of improvement capability and its relationship with organisational performance, which suggests that the concept of improvement capability is useful for understanding quality improvement and improving and sustaining the performance of healthcare organisations.

Funding
This work was supported by the Health Foundation in the UK [grant number 7426]. The Health Foundation was not involved during the design, delivery or submission of the research.