A scoping review of the use of quality improvement methods by community organizations in the United States, Australia, New Zealand, and Canada to improve health and well-being in community settings

Abstract Background Health-care facilities have used quality improvement (QI) methods extensively to improve quality of care. However, addressing complex public health issues such as coronavirus disease 2019 and their underlying structural determinants requires community-level innovations beyond health care. Building community organizations’ capacity to use QI methods is a promising approach to improving community health and well-being. Objectives We explore how community health improvement has been defined in the literature, the extent to which community organizations have knowledge and skill in QI and how communities have used QI to drive community-level improvements. Methods Per a published study protocol, we searched Scopus, Web of Science, and Proquest Health management for articles between 2000 and 2019 from USA, Australia, New Zealand, and Canada. We included articles describing any QI intervention in a community setting to improve community well-being. We screened, extracted, and synthesized data. We performed a quantitative tabulation and a thematic analysis to summarize results. Results Thirty-two articles met inclusion criteria, with 31 set in the USA. QI approaches at the community level were the same as those used in clinical settings, and many involved multifaceted interventions targeting chronic disease management or health promotion, especially among minority and low-income communities. There was little discussion on how well these methods worked in community settings or whether they required adaptations for use by community organizations. Moreover, decision-making authority over project design and implementation was typically vested in organizations outside the community and did not contribute to strengthening the capability of community organizations to undertake QI independently. Conclusion Most QI initiatives undertaken in communities are extensions of projects in health-care settings and are not led by community residents. There is urgent need for additional research on whether community organizations can use these methods independently to tackle complex public health problems that extend beyond health-care quality.


Introduction
As we continue to understand the role that social determinants of health play in affecting population health and well-being outcomes, the need to build capacity for systematic improvement in communities where people 'are born, grow, live, work and age' has never been more urgent. Initiatives such as Robert Wood Johnson Foundation's Culture of Health Action Framework [1] and CDC foundation's Thriving Together initiative [2] have enumerated the complex, interrelated dimensions of community health and well-being-e.g. health-care access, affordable housing, transportation, and poverty reduction-that must be addressed simultaneously for communities to thrive. Public Health 3.0the US Department of Health and Human Services definition of the modern era of public health practice that emphasizes cross-sectoral collaboration to address the social determinants of health [3]-recommended shifting the focus of community public health efforts from being owned and delivered by public health agencies to being led by diverse community-based coalitions focused on local priorities and contexts.
1. How has community health improvement been defined? 2. What QI approaches have been used for community health improvement? 3. How are these approaches similar or different from those implemented in clinical settings (health-care improvement)?

Methods
We used Batalden & Davidoff's definition of QI: a 'systematic approach to improve outcomes and systems by building the capability of communities to identify, prioritise and develop solutions to local systems problems' [6]. Table 1 lists operational definitions of other key terms [7]. We used Arksey and O'Malley's scoping review framework [8] with Levac, Colquhoun, & O'Brien's proposed enhancements to conduct this review [9]. Our review protocol, in BMJ Open, is available at https://bmjopen.bmj.com/content/9/12/e034302. Because these review method details are published, we present an abridged account here. Our research team was comprised of a faculty member and three students (two doctoral and one undergraduate) in the School of Public Health with years of experience in QI practice and community health improvement.

Inclusion and exclusion criteria
We reviewed peer-reviewed articles published in English from the USA, Australia, New Zealand, and Canada. We limited our review to these countries because of their similar national contexts. They are high-income countries that are part of the Anglosphere, with liberal market economies (in contrast to continental Europe's more coordinated market economies), and that experience health disparities between their White/Caucasian racial majority and their minority including indigenous populations [10]. We considered studies published between 2000 and December 2019 because the use of systematic QI methods to improve health was limited prior to 2000, as the Institute of Medicine published the 'Crossing the Quality Chasm' report in 2001 that defined the six pillars of high-quality health care [11]. We placed no restrictions on study type. To be included, studies had to use QI approaches to address community-level well-being outcomes or a community's capacity to improve in a community setting. Note that we did not place requirements on 'who' carried out the An activity, evidence-based program or policy that took place (i.e. is not merely proposed) QI approach Any QI method, such as Lean, PDSA, Six Sigma, or the Model for Improvement, or description of systematic process to improve community well-being Exclusion criteria: Article focuses on drivers of improvement, effectiveness of improvement, etc., but does not use QI approach or describe QI processes. Article describes approaches to improve community, coalition, or program function (e.g. improve attendance of coalition members at meetings) without linkage to community well-being outcome. Intervention took place within the walls of a facility with no linkage to community setting. improvement work (e.g. community organization, community members, and institutions)-rather, this question was part of our findings. We excluded studies that (i) described interventions to improve quality but did not report using a systematic improvement method; (ii) did not focus on improving community health or well-being outcomes (e.g. study outcomes were improving program function, such as meeting attendance, without connection to a community well-being outcome, such as food security); and (iii) described QI efforts or interventions undertaken within a facility (e.g. a clinic) rather than in the community. Table 2 shows inclusion and exclusion criteria [7].

Data sources and search keywords
We identified relevant studies through Scopus, Web of Science, and Proquest Health Management databases. Our search strategy combined terms in three categories: (i) 'community organizations' (e.g. community coalitions or health departments); (ii) 'QI methods' enumerated by commonly used terms describing systematic QI approaches; and (iii) 'health and well-being,' described by terms including education, justice, and equity. Our protocol paper lists the complete search string details and justification for selecting data sources. We hand-searched references of studies we deemed relevant during full-text screening.

Study selection
Our study selection involved three phases. In phase one, three authors (MWT, TC, and RR) reviewed 2% of titles and abstracts from extracted articles using the final search criteria. Using the inclusion criteria in Table 2, we designated studies as 'eligible,' 'ineligible,' or 'maybe' for full-text review. As we progressed through the 2% of title and abstracts, we discussed discrepancies in designations and adjusted interpretations of inclusion criteria. By completion of the review of the 2% of titles and abstracts, we reached an inter-rater reliability >80%. In phase two, one reviewer (RJ) reviewed the remaining titles and abstracts using the same inclusion criteria and designation strategy. Studies without abstracts were designated as 'maybe' if titles did not warrant immediate exclusion. In phase three, two authors (MWT and RJ) reviewed the full texts of each abstract designated as 'eligible' or 'maybe,' using the exclusion criteria to decide whether to exclude the study and documenting the reason. Through regular meetings with a third author (RR), we reached a consensus about studies where decisions on inclusion or reasons for exclusion differed.

Data extraction and charting
We created the charting form after extracting data from the first few studies through consultations with the research team. We determined that identifying the role of the community and articulating the extent to which community members actively participated in study design or implementation were important. Therefore, one author (RR) created a customized data-extraction template that specified the institutional (i.e. university, government, or private organization) and community (i.e. community-based organization/individual, local health department/agency, or school) partners associated with the study and their roles.
We also reviewed the literature on collective impact [13] and Arnstein's ladder of citizen participation [14] to develop meaningful categories to specify the locus of decisionmaking authority in each study. We created four categories: (i) institutional organizations (defined above), (ii) community organizations, (iii) multisectoral partnerships (multiple organizations and sometimes community residents collectively working toward an outcome), or (iv) community residents. Two authors (MWT and RJ) extracted the data; one author charted the data, and the second reviewed and amended the data with additional information or revisions in interpretation. Disagreements were resolved in regularly scheduled author meetings.

Data synthesis and presentation
Data synthesis involved qualitative and quantitative components. We presented summary counts of included and excluded studies using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flowchart and graphically summarized study characteristics (e.g. health area focus and QI method used). We recorded and tabulated each study's community role and locus of decision-making authority. We presented data syntheses in tabular form.
In addition, three authors (MWT, RJ, and RR) independently synthesized the findings across studies to answer the research questions. We followed Braun, Clarke, Hayfield, & Terry [15] guidelines for thematic analysis (TA), that researchers should 'use the approach to TA that is most appropriate for their research,' and 'use it in a "knowing" way' to 'produce an overall coherent piece of work' (p. 7). First, each author individually listed salient themes from an integrated review of studies. Then, through consultation, we synthesized individual themes to identify overall findings and identified what is missing in published literature to set future research priorities.

Results
Of the 10 088 unique articles identified through our database search, we deemed 9965 irrelevant during abstract/title screening ( Figure 1). We initially selected 123 for full-text review. Within this set, we excluded 91 (45 for not using a systematic QI approach, 29 for not taking place in a community setting, and 17 for not targeting community well-being outcomes). We ultimately selected 32 studies for data extraction, listed in Table 3. Salient characteristics are summarized in Figure 2.

Characteristics of studies
Geography, settings, and focus areas Figure 2 shows that nearly all the studies were set in the USA, encompassing 19 states and a wide geographic distribution. Interventions were implemented in a wide variety of community settings including Boys and Girls Clubs, YMCAs, home visits, indigenous communities, and low-income neighborhoods. The target groups for a significant majority of the studies were low-income and minority populations and emphasized mothers, youth, and adolescents. One study focused on the elderly, and one on indigenous communities.
Study focus areas split between those seeking to improve community health through prevention or promotion activities (19 studies) versus through chronic disease management (13 studies). Both groups included a diverse set of health topics and target populations across the lifespan. Examples of prevention projects included adolescent sexual health, substance abuse prevention, food insecurity, smoking cessation, adolescent mental health, immunization, breastfeeding, well baby care, healthy aging, and intimate partner violence. Chronic disease management topic areas were childhood obesity, substance abuse, diabetes, and pediatric asthma.

Interventions and use of QI
Many studies used QI methods to generate solutions (frequently community outreach) to improve implementation of Box 1.
• Diabetes education: Diabetes self-management education (DSME): a public health, system-level intervention to improve glycemic control in adults. In six local health departments, facilitators trained a QI team and helped them develop and implement a 9-to 12-month QI project in their community to improve DSME services [19]. • Early childhood care: HealthMPowers is a 3-year early care and education (ECE) program that uses continuous improvement to provide training, improve programs, measure impacts, and sustain partnerships. Sixty-five ECE centers in Georgia formed a team that implemented annual self-assessments and improvement plans, such as improving home environments [20]. • Pediatric asthma: The Asthma Improvement Collaborative enhanced pediatric asthma care, e.g., by strengthening community and hospital relationships. A multidisciplinary improvement team developed a key driver diagram of emergency department use by the target population and tracked outcomes using control charts [29].
programs, guidelines, or standards. A few used QI to develop local interventions. While project team members received QI training in most studies, the training objective was to apply project-specific QI methods, rather than to build general QI expertise that could apply to other community improvement efforts. Box 1 shows typical QI use examples.

Institutional and community roles
Nearly all selected studies relied on external institutional partners (e.g. university, technical service provider, or federal or state agency) for funding, planning, training, supervision, and/or evaluation. Community organizations (e.g. YMCAs, schools, and local health departments) were involved in 31 of the 32 studies but did not always have decision-making authority and often were involved only in implementing interventions. Moreover, since community organizations typically were local chapters of state or national institutions (e.g. YMCA), the extent to which the local chapters truly were integrated into and reflect the local community was not always clear. Table 3 describes the distribution of decisionmaking authority across studies. Institutional partners had decision-making authority over priorities and interventions in 13 studies, community organizations in 12, and multiple stakeholder organizations (which could include community organizations and community residents) in 5. Only 2 studies were designed to 'center' decision-making authority about interventions directly within the community.

QI methods and research designs
QI research study designs varied in rigor and in the types of designs used. Three studies used randomized designs. Most used quasi-experimental designs of varying strength: six used comparison groups, five used interrupted time series, and six used pre-post designs. Ten studies used narrative descriptions of projects. Two employed mixed methods. Overall, detailed information about how QI study activities were implemented was lacking.

Outcomes
As Table 3 shows, 18 studies used project-relevant outcome measures (e.g. related to sexual behavior, emergency department visits, hypertension control, cervical cancer screening, and breastfeeding behavior). Some of these studies also used process variables proximal to the measured outcomes, such as availability of sugar-sweetened beverages, attendance at diabetes self-management classes, or satisfaction with obesity prevention programs. Fourteen studies exclusively used process measures, including implementation variables such as the number of workshops conducted or the number of sites conforming to performance standards. Five studies did not report any results. Twenty-five of 27 reported positive change at the end of the QI interventions; two studies reported null results between intervention and comparison groups. Because statistical analysis of outcomes was sparsely reported, it was not possible to assess whether positive results that were reported were significant, could be attributed to the intervention, or reflected selective reporting by the authors.

Principal findings
We report our principal findings by the research questions described earlier.
1. 'How has community health improvement been defined?' All the studies defined health improvement in terms of management of chronic diseases or health promotion activities. This focus is substantively different from improving the quality and safety of patient care, which has been the primary emphasis of QI to date in the health sector. Moreover, the studies described complex, multifaceted interventions that involved education, behavior change, and modifications to service delivery processes. This has not historically been the focus of QI initiatives in clinical settings, which are more narrowly focused on clinical interventions. The current growth and interest in Learning Healthcare Systems [48] and in Learning Health Networks [49] that enable collaborations between patients, families, and care teams to address the entire system of care for a patient have begun to shift this paradigm in the healthcare space, but the emphasis is still on providing care after patients have been diagnosed. The health promotion or public health aspects of some of the included studies differentiate the notion of 'improvement' in community settings. 2. 'What QI approaches have been used for community health improvement?' The Model for Improvement (MFI) [50] was the most common improvement method, mentioned in five studies. Fourteen studies mentioned the use of Plan-Do-Study-Act (PDSA), although some of these may have used PDSA and MFI as synonyms. Breakthrough collaboratives or other learning networks were used in six studies. Individual tools such as driver diagrams [51], flowcharts, run charts [52], and cause-and-effect diagrams also were mentioned, as shown in Figure 2. Scant detail was provided on how exactly the QI methods were used. Eight studies left specific QI methods, approaches, or tools unstated. 3. 'How are these approaches similar or different from those that have been implemented in the clinical setting (health-care improvement)?' No new methods were developed specifically for community health improvement. Several studies applied health-care QI methods to complex, multicomponent interventions. However, there was little discussion on how well these worked in community settings or how to adapt health-care methods for typically encountered community setting situations (e.g. no routinely collected electronic medical record data; no clearly defined protocols for interventions; QI teams that are coalitions and not employees of clearly defined health systems). Overall, comparison between community and health-care QI methods was challenging because of the lack of detail about how QI activities were implemented in the included studies, which is a common problem in QI studies [53].

Strengths and limitations
To our knowledge, this review is the first to study the use of QI methods in community settings. However, because these settings are not clearly defined, we needed to create operational definitions for what constituted community improvement, and the studies we selected were based on these definitions. Other definitions for community health improvement may result in other studies being included. Moreover, our study only included peer-reviewed literature. It is possible that community organizations are engaged in QI projects that have been documented in websites, donor reports, or conference presentations that have not reached academic journals. Conducting a similar review including the gray literature would likely produce a larger body of work than we have identified in this review.
Interpretation within the context of the wider peer-reviewed literature Since QI for health-care improvement is a mature field, we expected to identify a body of literature demonstrating how QI researchers have adapted these methods for use in more complex, distributed, and data-poor community settings. Our selected studies failed to address the complex nature of community health in two critical ways. First, while our search criteria intentionally included articles addressing both health and well-being, most of the studies emphasized only physical aspects of health. They were conceptualized as extensions of hospital-based QI efforts that focus on improving clinical outcomes or enhancing operational care delivery processes. The World Health Organization recognizes that community well-being extends beyond physical health and includes mental and social aspects [54]-all of which should be the scope of community health improvement. Second, while most of the studies focused on low-income and socially disadvantaged populations, few addressed the social determinants of community health or explicitly acknowledged structural factors that affect outcomes. These factors include income inequality, mass incarceration, and structural racism [56]. Papers that did focus on structural factors were Brimblecombe et al. [22], which addressed system drivers of food insecurity, and Inkelas, Bowie, and Guirguis [40], which described a network of organizations using QI to improve population outcomes such as child well-being through multisectoral collaboration.

Implications for practice, policy, and research
Our findings indicate the need for more research on the applicability of QI methods on the social determinants of health and well-being in allied systems such as education and housing. We must build knowledge about how to define and measure outcomes, collect process data, and test and implement interventions to tackle these complex problems.
We also must learn how to engage community residents with deep local knowledge as an integral part of community improvement efforts; the predominantly top-down approaches we found in this review may impede improvement in underserved and marginalized communities. QI teams in community settings must be assembled, organized, and managed differently from clinical teams. There is little peerreviewed, academic literature about how this should be done.
Involving community members should include much more than just assembling teams. Community-led QI initiatives should be based on principles of Collaborating for Equity and Justice [57], with the goal of building resident leadership to enable community members to set an improvement agenda focused on systems' change, not just unitary outcomes. These principles are echoed in other community-led, equity-based approaches such as communitybased participatory research and design justice, with the tenet of 'nothing about us without us' [58]. Embedding QI capabilities into communities should be an intentional focus of community health improvement efforts and is an area of research that is not reflected in the peer-reviewed literature.

Implications for future documentation
Finally, this study shone light on a potential gap between improvement work that may be undertaken by communities and what is published in peer-reviewed literature. As we have indicated, the 32 peer-reviewed papers that met our inclusion criteria document studies that have been led by researchers and academic implementers because these are the ones with the resources and incentives to engage in formal documentation efforts and the peer review. Community-led improvement initiatives that may have been documented locally as project reports or as presentations for stakeholders would not have made it into the peer-reviewed literature that we reviewed and could represent a bias in our findings. To expand the documentation of community-based efforts, accessible methods need to be developed for communities to synthesize and report on findings and learning. A recent example of such an effort is the participatory synthesis process that was used in the Robert Wood Johnson Foundation-funded 100 Million Healthier Lives initiative in which community implementers partnered with evaluation team to document generalizable insights from routine program data [55,59]. The process of synthesis, documentation, review, and publication in peer-reviewed journals was arduous and time-consuming and required a commitment well beyond the funds provided by the grant. To accelerate and facilitate the process of dissemination from the field, journals need to create accessible and inexpensive options for dissemination. While a few journals have begun to publish field reports (e.g. BMJ's Quality Improvement Reports), the submission process has an academic focus that many community practitioners may find burdensome and not worth the effort.

Conclusion
Public health has recognized the need to go beyond its traditional boundaries and to engage cross-sectoral collaborations to address social determinants of health. Our scoping review indicates that few published community health improvement initiatives extend beyond single-population health outcomes to address multifaceted systems' change. Details are scarce about how to adapt existing QI methods to these contexts or whether new methods should be created. More importantly, decisions to use QI methods for community health are not yet in the hands of community members. As the coronavirus disease 2019 era has shown, common restrictions imposed at the state or county level result in widely varying results at the community level [60]. While communities are subject to the same constraints, their infection processes are widely different and therefore require different, context-appropriate containment solutions. Communities urgently need to be actively involved in developing solutions to improve health and well-being. Our scoping review shows that community health improvement that has been published in peer-reviewed literature is still primarily focused on providing clinical care in community settings, with some progress in implementing interventions that reach whole populations-a finding that may reflect bias in what gets published rather than work happening on the ground. There is much work to be done.