Adapting Strategies for Optimal Intervention Implementation in Nursing Homes: A Formative Evaluation

Abstract

Background and Objectives

Nursing homes pose unique challenges for implementation of research and quality improvement (QI). We previously demonstrated successful implementation of a nursing home-led intervention to improve relationships between frontline staff and residents in 6 U.S. Department of Veterans Affairs (VA) Community Living Centers (CLCs). This article discusses early adaptations made to the intervention and its implementation to enhance frontline staff participation.

Research Design and Methods

This is a formative evaluation of intervention implementation at the first 2 participating CLCs. Formative evidence—including site visitors’ field notes, implementation facilitation records, and semistructured frontline staff interviews—were collected throughout the study period. Data analysis was informed by the Capability, Opportunity, Motivation, and Behavior model of behavior change.

Results

Adaptations were made to 5 a priori intervention implementation strategies: (a) training leaders, (b) training frontline staff, (c) adapting the intervention to meet local needs, (d) auditing and providing feedback, and (e) implementation facilitation. On the basis of a 6-month implementation period at the first CLC, we identified elements of the intervention and aspects of the implementation strategies that could be adapted to facilitate frontline staff participation at the second CLC.

Discussion and Implications

Incremental implementation, paired with ongoing formative evaluation, proved critical to enhancing capability, opportunity, and motivation among frontline staff. In elucidating what was required to initiate and sustain the nursing home-led intervention, we provide a blueprint for responding to emergent challenges when performing research and QI in the nursing home setting.

Long-term care, Methodology, Organizational and institutional issues, Qualitative research methods, Veterans

Nursing homes are reputed to be challenging environments for implementing research and quality improvement (QI) programs (Nazir, Levy, Rudolph, & Unroe, 2017; Quadagno & Stahl, 2003; Siegel & Paterniti, 2015). Unique considerations, such as top-down flow of communication, staff time constraints, task-orientation, and high turnover among frontline staff, may lead to suboptimal implementation or maintenance of innovative practices (Berlowitz et al., 2003; Cranley et al., 2018; Maas, Kelley, Park, & Specht, 2002; Rantz et al., 2012). A promising solution to improve health care delivery in long-term care is to assess and support organizational readiness for behavior change (Weiner, 2009). Considered a critical precursor to successful implementation of complex change in health care (Amatayakul, 2005), an organization’s readiness for change is a function of organizational members’ shared commitment to implementing change and a shared belief in their collective capability to do so. Recent work has thus focused on equipping nursing home staff with opportunities and skills needed to successfully engage in implementation of research interventions and QI (Pimentel et al., 2019; Wiener, 2003). Department of Veterans Affairs (VA) nursing homes (i.e., community living centers [CLCs]), for example, have had success in implementing innovative promising practices through employing strengths-based learning, observation, relationship-based teamwork, and efficiency (Hartmann et al., 2018).

Conceptual Framework

To operationalize promising implementation practices for widespread use, we developed the Learn from the bright spots, Observe, Collaborate in huddles, Keep it bite-sized (LOCK) framework and facilitation package: (a) learn from the bright spots, (b) observe, (c) collaborate in huddles, and (d) keep it bite-sized (Mills et al., 2018; Table 1). We relied on blended facilitation—that is, a team-based approach that leverages the complementary skills and expertise of implementation agents who are external and internal to the implementation setting—to overcome known barriers to sustaining research and QI programs in nursing homes (Pimentel et al., 2019). We coupled LOCK with an intervention grounded in the Capability, Opportunity, Motivation, and Behavior (COM-B) model of behavior change (Michie, Atkins, & West, 2014; Michie, van Stralen, & West, 2011). COM-B has been widely used to guide implementation work, including in long-term care (Fleming, Bradley, Cullinan, & Byrne, 2014; Peiris et al., 2015). The model posits that individuals’ behaviors are influenced by—and reciprocally influence—three main components that also interact with each other: the physical and psychological capability to enact a behavior, the existence of an opportunity to enact a behavior, and having the internal or external motivation to enact a behavior (Figure 1). This implies, for example, that achieving an outcome such as more frequent staff interaction with residents necessitates increasing the influence of one or more of the COM components. This may involve teaching staff about the importance of interaction for improving quality of care by giving examples of how care improves when staff and residents develop strong relationships (improving capability). It may mean that supervisors support frontline staff by pitching in to help with tasks to increase the time for meaningful interactions (improving opportunity). It may also involve encouraging residents to speak up during interdisciplinary care team meetings about specific examples of recent interactions with staff and what they meant to them (improving motivation).

Table 1.

Elements of the LOCK Intervention and Their Definition

LOCK intervention elements	LOCK element definition
Learn from the bright spots	Identify desired behaviors or positive deviants to spread throughout the nursing home
Observe	Everyone in the nursing home conducts 5-min observations of the environment, documenting bright spots
Collaborate in huddles	Brief stand up meetings to share bright spots, evaluate data, and plan for next steps
Keep it bite-sized	Limit all intervention activities to 5- to 15-min increments and ensure they are easily integrated into existing routines
Facilitation	Actions that help stakeholders gain the knowledge, skills, and confidence to implement a new practice

Note: LOCK = Learn from the bright spots, Observe, Collaborate in huddles, Keep it bite-sized.

Figure 1.

The Capability, Opportunity, Motivation, and Behavior (COM-B) model of behavior change.

LOCK-Based Intervention and Its Implementation

We used LOCK and COM-B to implement QI projects that were nested within a formal research study. The study, which took place in six CLCs, assessed the effectiveness of a LOCK-based intervention to improve frontline staff interactions with CLC residents and, ultimately, resident engagement. This study is detailed elsewhere (Hartmann et al., 2018). Briefly, CLC leadership selected one or more neighborhood-level QI projects focused on the following outcomes: (a) frequency of staff–resident interactions, (b) quality of staff–resident interactions, and (c) frequency of resident engagement in meaningful activity. Researchers trained and continually supported staff on the use of the LOCK framework, which the frontline staff then used to implement their selected project. Staff performed systematic observations relevant to their neighborhood’s QI project, using standardized observation tools (Hartmann et al., 2017) to collect data on “bright spots,” that is, positive examples of staff and resident behavior. Staff then discussed their observations in brief “bright spots frontline huddles” that emphasized problem solving, collaborative communication, and identifying and building upon areas of strength. Implementation of the LOCK-based intervention overall was significantly associated with increased staff communication with CLC residents during provision of direct care (β = 0.083, 95% confidence interval [CI]: 0.04–0.126, adjusted p = .0012) and with decreased negative staff interactions with residents (β = −0.035, 95% CI: −0.062 to −0.009, adjusted p = .0288; Hartmann et al., 2018). We also demonstrated that LOCK’s inherently motivational features were critical to achieving successful implementation and promising outcomes of the staff-led intervention (Mills et al., 2019).

Little is published on how to achieve successful intervention implementation in long-term care. One key aspect of successful implementation is intervention adaption to local context. Provided that essential elements of an intervention are preserved, intervention adaptions can increase their relevance and applicability for a target population or setting (Durlak & DuPre, 2008). Strategies for intervention implementation may shift over time. The implementation science literature is rich with examples supporting the need for such flexibility to enhance adoption and uptake of research or QI in clinical settings (Chung, Mikesell, & Miklowitz, 2014; Mignogna et al., 2018). But, to the best of our knowledge, literature is lacking around how adaptations are made over the course of a project. A deep-dive into the reality of research and QI processes in nursing homes from the perspective of the researchers themselves may offer examples to consider when attempting to implement research, clinical innovations, and QI in this setting.

This article describes early adaptations that we VA researchers made in response to emergent challenges during the implementation of a CLC-based research intervention and provides lessons learned for improving study participation among frontline staff. We used an implementation- and progress-focused formative evaluation approach, which is helpful for understanding more fully the major barriers to intervention implementation and what is required to achieve study goals (Stetler et al., 2006). Here, we highlight how the intervention was implemented per study protocol at a pilot CLC, what we learned from that implementation process, and how we applied COM-B to refine our implementation strategies for the use at the second CLC. We focus on the first two CLCs because our approach to implementing the intervention in the second CLC represented our approach for the final four CLCs in our study.

Methods

The research study was approved by the VA Central Institutional Review Board and the research and development committees of Edith Nourse Rogers Memorial Veterans Hospital, Boston VA Healthcare System, and Tuscaloosa VA Medical Center.

Setting and Study Timeline

The first two CLCs in the study, CLC A and CLC B, are in the Southeastern United States. At the time of our study, CLCs A and B were comparable in terms of their bed size (112 vs 97 beds, respectively). In CLC A, facility leadership invited nursing staff from three nonhospice neighborhoods to participate, whereas leadership in CLC B engaged both nursing and dietary staff from two neighborhoods that provide respite care primarily for veterans with dementia.

Researchers spent 10 weeks (February to April 2015) preparing for the first of the three site visits and engaging in preimplementation facilitation in CLC A. The LOCK-based intervention was implemented over the course of 25 weeks (April to December 2015), using five discrete COM-B-related strategies (Powell et al., 2015; Table 2).

Table 2.

LOCK Intervention Implementation Strategies and Targeted Domains, per the Capability, Opportunity, Motivation, and Behavior (COM-B) Model

Implementation strategies	Capability	Opportunity	Motivation
1. Identify and prepare champions for implementation	✓	✓	✓
2. Empower champions to train and motivate staff	✓	✓	✓
3. Collaborative tailoring of the LOCK-based intervention		✓
4. Audit and feedback	✓	✓
5. Blended facilitation from research team	✓		✓

Note: LOCK = Learn from the bright spots, Observe, Collaborate in huddles, Keep it bite-sized.

Before any interaction with CLC B, we reviewed our experience at CLC A and made adaptations to our implementation strategies. The preintervention and intervention periods for CLC B began while implementation at CLC A was ongoing. Preimplementation activities with CLC B commenced in April 2015 and continued for 8 weeks. The intervention period for CLC B spanned 39 weeks (June 2015 to March 2016).

Data Sources

We collected a broad base of qualitative formative evidence at the two study sites to understand the context in which the LOCK-based intervention was implemented. Formative evaluation data comprised the following: site visitors’ field notes, facilitation records, and interviews of CLC participants. First, researchers who performed in-person site visits took hand-written field notes and later met multiple times as a group to summarize their impressions in a comprehensive master document. Second, an external facilitator recorded various aspects of the implementation process, including dates of contact with the CLCs’ internal facilitators and open-ended comments about facilitators and barriers to implementation as indicated by internal facilitators during phone calls and E-mails or as noticed by the external facilitator. Third, we conducted in-person semistructured qualitative interviews with internal facilitators and participating CLC staff during the final site visit. Staff were invited via E-mail and in-person contact to participate in interviews. Eligible participants included CLC leadership, clinicians, nurses, nursing assistants, social workers, physical/occupational therapists, psychologists, pharmacists, dietary staff, chaplains, and others whose jobs were related to QI. COM-B guided development and organization of the interview guide, which elicited experiences with the LOCK-based intervention; how the intervention influenced job performance, resident care, and the CLC environment; and impressions of huddles and bright spot observations. Follow-up probes focused on understanding specific COM-B elements. Researchers recorded each interview. A fourth source of data—templated weekly paper reports completed by CLC staff and internal facilitators—was used as a proxy measure of implementation uptake.

Data Analysis

We focused on the QI project implementation at the first two sites to uncover the process of intervention adaptation and participant engagement. COM-B provided the conceptual foundation for the evaluation. Within that, we focused on the researcher behaviors that were designed to positively influence CLC frontline staff members’ capability, opportunity, and motivation.

Three researchers independently coded the field notes and facilitation records using COM-B-based a priori codes about researcher behaviors. Example codes include “increasing capability and opportunity by simplifying or modifying tools,” “increasing meaning/motivation by simplifying or modifying tools,” and “addressing capability/opportunity by adding training” (Supplementary File 1). Coders met to discuss codes, coming to consensus via discussion.

Six researchers used a rapid analysis approach (Gale et al., 2019) and a second set of a priori codes based on COM-B and intervention aspects (Supplementary File 2) to analyze the semistructured staff interviews. We first constructed an analytic matrix that created an intersection of the COM-B domains and site and researcher behaviors (columns) and LOCK intervention elements (rows). Next, we assigned each audio-recorded interview a primary and secondary coder. Coders listened iteratively to interview audio recordings, summarizing content and transcribing quotes into relevant cells in the matrix; data could appear in more than one cell. After completion of coding of all interviews from a site, the team met to review the data and come to consensus on any coding discrepancies.

We performed counts of the weekly paper reports that CLC staff and internal facilitators completed.

Results

Results are organized by discrete implementation strategy and summarize findings from site visitors’ field notes, facilitation records capturing perspectives of external and internal facilitators, and interview data from five staff members in CLC A and 23 staff members in CLC B. Our chronological narrative details “per protocol” pilot implementation in CLC A and adjustments made to enhance capability, opportunity, and motivation to participate in the study (Table 3).

Table 3.

Adaptations Made to LOCK Intervention Implementation Strategies in CLC B and Targeted Domains, per the COM-B Model

Intervention adaptations in CLC B	Capability	Opportunity	Motivation
Training leaders
• Researchers decreased presentation material for preimplementation phone calls to enable discussion	✓	✓	✓
• Site visitors helped CLC stakeholders develop a QI project		✓	✓
• Researchers held focused training on bright spots huddles after (vs before) initial site visit	✓
Training frontline staff
• Internal CLC point of contact engaged nurse managers to conduct bright spots frontline huddles	✓	✓	✓
Site visitors “kicked off” intervention at initial visit			✓
• Researchers held face-to-face staff training, supported in-person interactions with multimedia, and standardized training	✓	✓	✓
• Researchers engaged a nurse educator to continue in-person training of CLC frontline staff	✓	✓
Tailoring the intervention to meet local needs
• Researchers developed a fill-in-the-blank “menu” for CLC stakeholders to design their QI project		✓
• Researchers incorporated 6-week PDSA cycles to trial observation tools/QI projects		✓
Auditing and providing feedback
• Researchers shortened tool-user reports and attached them to observation tools	✓	✓
• Researchers shortened facilitator reports to focus on bright spots frontline huddles	✓	✓
• Researchers highlighted potential discussion points, simplified report content, and provided longitudinal data	✓	✓
Implementation facilitation
• External facilitator simplified communication and provided moral support to internal facilitators	✓		✓
• Researchers shared best practices among CLCs to cultivate learning communities	✓		✓
• External facilitator engaged alternate internal facilitators when needed	✓		✓

Note: CLC = Community Living Center; COM-B = Capability, Opportunity, Motivation, and Behavior; LOCK = Learn from the bright spots, Observe, Collaborate in huddles, Keep it bite-sized; QI = quality improvement; PDSA = plan–do–study–act.

Training Leaders to Improve Capability, Opportunity, and Motivation

Piloting in CLC A

Members of the research team, including the study’s principal investigator (C. W. Hartmann), site visitors (A. L. Snow, W. L. Mills, N. J. Wewiorski), and the external facilitator (J. A. Palmer), scheduled individual phone calls in February 2015 with our CLC partners, the facility’s associate chief nurse (i.e., the study point of contact), and nurse managers and assistant nurse managers from the three neighborhoods (i.e., internal facilitators). These preimplementation “getting to know you” meetings enabled us and our CLC partners to learn more about one another, discuss areas of strength and sources of stress for frontline staff, and brainstorm ideas for using the study’s observation tools to ameliorate identified stressors (capability, opportunity, and motivation). The initial phone call with the study point of contact further focused on strategies to achieve optimal intervention implementation (capability), such as engaging staff unions in the implementation process or formally recognizing staff participation in annual performance evaluations.

Follow-up phone calls with internal facilitators followed “train, do, and review” cycles. We held group calls in early March 2015 to train the study point of contact and internal facilitators on the study’s observation tools and plan a “kick-off” event for them to introduce the study to CLC staff prior to our initial site visit. This first training call focused on how internal facilitators could market the study to frontline staff to improve motivation and provided detailed guidance on performing a bright spots frontline huddle. We then asked internal facilitators to hold their first bright spots frontline huddle with frontline staff. We followed up with one-on-one calls to review the internal facilitators’ kick-off events and answer remaining questions about the observation tools. A second “train, do, and review” cycle beginning in late March 2015 focused on one of the study observation tools (the Realized Opportunity for Relationship [ROR] tool to standardize and quantify observations of CLC staff interactions with residents; Hartmann et al., 2017) and how internal facilitators could use a bright spots frontline huddle to orient frontline staff to using the ROR tool (capability). By the end of the preimplementation period in April 2015, we expected CLC leadership to select their QI project, that is, decide whether their neighborhood would focus on improving (a) frequency of staff–resident interactions, (b) quality of staff–resident interactions, or (c) frequency of resident engagement in meaningful activity.

Refining in CLC B

We learned from CLC A’s internal facilitators that “doing [too many] things too early” overwhelmed study participants and adversely affected their participation. We therefore revisited COM-B and made significant changes to our preimplementation series of phone calls with our study partners at CLC B. First, we decreased the material presented during each phone call to enable more time for discussion (capability, opportunity, and motivation). For example, we introduced each of the three observation tools over the course of three phone calls and spent an additional phone call focused on mindful observation to demonstrate the importance of the “O”—observe—in the LOCK framework. Second, rather than ask our CLC partners to develop their own QI project and then commit to using the LOCK-based intervention to support that project, we communicated that we would help them select a QI project at our site visit (opportunity/motivation). Third, owing to the primacy of the bright spots frontline huddle to the intervention, we waited until after the initial site visit to have a focused discussion with internal facilitators on how to conduct the huddles (capability).

Training Frontline Staff to Improve Capability, Opportunity, and Motivation

Piloting in CLC A

After the phone call series and prior to our first site visit, we provided PowerPoint slides that explained the study rationale and specific instructions for using all three observation tools (capability). This PowerPoint presentation was meant to be shown at an in-service session, during bright spots frontline huddles, or at shift change meetings. We provided motivational training videos in DVD format and via YouTube (accessible on the VA network) to supplement the in-person training. Internal facilitators were asked to show frontline staff the training video prior to our initial site visit.

At the initial site visit in April 2015, we learned that no staff member had received in-person training or viewed the supplementary videos. We therefore restructured the initial site visit in real time to comprise a third cycle of “train, do, and review,” with the two site visitors developing a new training strategy on the spot to provide direct training with frontline staff, improving their capability. The site visitors used the facility’s portable DVD player as well as facility computers to show the motivational training video many times in many locations over the 3-day site visit and enable frontline staff to view it at their convenience. Site visitors worked flexibly to catch individual staff members or small groups on the neighborhood floor whenever they had a few minutes between patient care tasks for review of hard copies of the PowerPoint presentation. The training was not standardized—one site visitor familiarized staff with all three observation tools, while the other provided in-depth and experiential training on the use of one of the tools. After the initial site visit, we developed additional training videos specific to the observation tools that CLC A ultimately selected.

Refining in CLC B

Adaptations to the LOCK-based intervention and its implementation in CLC B were influenced both by our experience working with CLC A and by our study point of contact in CLC B, the nurse educator. We learned from CLC A the challenges of engaging already-busy clinical staff to lead QI efforts.

It was helpful to have someone outside of direct resident care be the point of contact [in CLC B] because that person had more time for the project. (Researcher)

The point of contact and her student assistant performed most study activities, including ongoing engagement and training of new frontline staff to participate in the intervention.

There was a thought that everyone would get trained on tools, including the night shift. This was the nurse educator’s initiative, because she was used to going in at night. (Researcher)

To support the CLC’s capability, opportunity, and motivation to sustain the LOCK-based intervention after the formal study period, we encouraged the point of contact to engage nurse managers to conduct the bright spots frontline huddles.

We learned after our initial visit to CLC A that it is the responsibility of the research team—and not solely our study partners—to generate motivation among frontline staff. We therefore used our initial site visit and individualized staff training to “kick off” the LOCK-based intervention. We increased staff members’ capability and opportunity to participate in the study using three strategies: first, by simplifying, clarifying, or modifying the training; second, by addressing skill development through additional training material and explanation; and third, by changing study processes to improve efficiency. We also influenced staff members’ motivation to participate in the study by making the training material easier to understand, practicable, and meaningful.

We learned the importance of first engaging face-to-face with study participants (motivation), using multimedia to support the initial interaction (capability/opportunity), and standardizing our training process for CLC B (capability). Site visitors brought with them a portable DVD player to show the training video and hard copies of a scripted PowerPoint training presentation. They used a current staff list provided by the study point of contact to identify and systematically train individual study participants. During the training, the site visitor read aloud the printed PowerPoint presentation slides, discussed the concept of “bright spots” and the goal of each observation tool, and practiced with the staff member using one of the observation tools on the neighborhood.

Site visitors also learned from their visit to CLC A that they would not be able to conduct in-person training with all staff members, especially those working the night shift. We therefore involved CLC B’s nurse educator early in the implementation period to enhance CLC staff capability and opportunity to participate.

[The] nurse educator was trained on how to train—this was her idea. She trained [individuals] about 5 or 6 times with the site visitors watching and giving feedback. (Researcher)

Site visitors tracked who viewed the training video, whether staff members were trained by site visitors and had on-the-unit practice, or if they would receive training later from the nurse educator.

Tailoring the Intervention to Meet Local Needs to Improve Opportunity

Piloting in CLC A

During our first site visit, we learned that CLC leadership had not selected their unit-level QI projects for application of the LOCK-based intervention. We spent extra time during and immediately after the site visit to help the study point of contact and internal facilitators select one observation tool for each participating neighborhood. We also adapted the intervention by empowering internal facilitators to switch their selected QI projects as needed over time to better meet frontline staff members’ priorities. Although only licensed nurses were initially involved in the QI project, we also encouraged CLC leadership to include certified nursing assistants in all study activities and bright spots frontline huddles.

Refining in CLC B

Additional local adaptations were needed to improve uptake of the LOCK-based intervention. For example, in addition to inviting all nursing staff to participate (as in CLC A) we worked with CLC leadership and study points of contact to engage nonclinical staff, such as those working in environmental management services and nutrition (opportunity).

To improve opportunity, we incorporated two new components into the intervention. The first was a “menu” that CLC B could use to develop their QI project. At the end of our first site visit, we asked study points of contact and internal facilitators for each participating neighborhood at CLC B to decide as a group the answers to the following questions: What observation tool will the neighborhood use? Who will complete the observation tool? How often will each person complete the observation tool? The second aspect we incorporated into the intervention was 6-week plan–do–study–act cycles for each unit to trial their selected observation tool. This enabled internal facilitators to more easily change the observation tool to better align with the goals and QI priorities of the participating neighborhoods.

Auditing and Providing Feedback to Improve Capability and Opportunity

Piloting in CLC A

We mailed hard copies of CLC A’s selected observation tools to the study point of contact and internal facilitators for distribution to participating frontline staff. Internal facilitators asked frontline staff in each neighborhood to use the observation tools to each conduct one 5-min observation per week and to complete a separate optional “tool-user report” each time they completed an observation. This paper-based report included 16 check-box items that asked frontline staff about their positive experiences, who they consult if they need guidance, and challenges with using the observation tool. Similarly, we asked internal facilitators to complete on a weekly basis a 17-item “facilitator report” that asked about recent experiences related to the bright spots frontline huddles or staff challenges in using the observation tool. We used counts of the tool-user and facilitator reports as proxy measures for intervention uptake. Both the tool-user and facilitator reports were available from an internal facilitator or kept at a central location, such as the nurses station, and then returned to centralized tool collection folders.

Owing to the duration of the implementation period in CLC A (25 weeks), we expected to receive at least 125 weekly tool-user and 25 facilitator reports. In the first half of the study, we received only 16 tool-user reports and four facilitator reports, indicating a need to improve capability and opportunity. To encourage more responses from CLC A, we shortened the tool-user report (from 16 to 9 questions focused on the same categories) and stapled it to the front of the observation tool. We also shorted the facilitator reports (from 17 to 13 questions) and revised the questions to focus only on the bright spots frontline huddles. Despite these modifications to lighten participant burden and enhance capability/opportunity, we received only one tool-user report and no additional facilitator reports in the second half of the study.

The structured observation tools, such as the ROR, were intended to point out to frontline staff examples of positive behavior occurring in their facility, which could then be shared and spread through the weekly bright spots frontline huddles. We learned from internal facilitators, however, that the data reports—that were meant to show patterns of bright spots in resident–staff interactions and resident engagement—were not meaningful to staff and therefore not actionable. Staff needed easily digestible reports of the aggregated observations: “If we can see where residents are most engaged, by time and place, then we can take action.” We therefore developed “audit and feedback” reports that were generated by frontline staff observations and showed graphical and tabular representations of at least 2 months’ worth of QI data by neighborhood, shift, and location in the CLC (capability).

Refining in CLC B

Owing to the duration of implementation (39 weeks), we expected to receive 195 weekly tool-user and 39 facilitator reports. A total of 36 tool-user reports and 16 facilitator reports were completed and returned in the first half of the study period. This, coupled with our experience with CLC A, taught us to simplify, clarify, and modify access to and the content of the reports to enhance staff members’ capability and opportunity to complete them during the remainder of the study period. After we employed a strategy to improve completion of these reports (as described above for CLC A), we received an additional 160 tool-user reports and four facilitator reports in the second half of the study period.

As in CLC A, we developed audit-and-feedback reports generated by frontline staff observations. Once the LOCK-based intervention was fully implemented, however, we learned that many of the observation tools that were returned to us were missing important information necessary for us to produce meaningful reports for the CLC staff. We therefore suggested that internal facilitators label their observation tool collection folders by shift. If staff members did not indicate the shift on the observation tool’s header before putting the tool into the collection folder, the folder labeling system acted as a failsafe.

At weekly check-in phone calls, internal facilitators reported that they could understand the data communicated in the audit-and-feedback reports, and that:

[W]hen things got done it was because some kind of communication of findings was happening. Huddling is key. The more you correctly huddle the better it will be. (Internal Facilitator)

However, site visitors at the mid-implementation visit learned that internal facilitators and frontline staff were not discussing the reports during bright spots frontline huddles because the report was “overwhelming in its volume.” To increase study participants’ capability and opportunity to use the reports, we highlighted and clarified potential discussion points and greatly simplified report content. First, we developed scripts for internal facilitators to follow when communicating report results during bright spots frontline huddles. Second, we provided a summary table with the prevalence of the outcome of interest and a brief description of that outcome. We also removed lengthy explanations of how the outcomes of interest were calculated.

[We] tried to make the data report more user-friendly and do some interpretation of the data for the site instead of asking them to do the interpretation for themselves. (Researcher)

Third, at the site’s request, we provided longitudinal, rather than cross-sectional, data capturing at least 3 weeks of staff observations. At a check-in call following our revisions to the data reports, internal facilitators reported that they were “happy with the revisions.”

Implementation Facilitation to Improve Capability and Motivation

Piloting in CLC A

The external facilitator (J. A. Palmer) had, at a minimum, weekly phone and E-mail communication with the internal facilitators. She adapted her communication style to fit the needs of the CLC partners. For example, she simplified the content of her E-mail communications when she realized that her internal counterparts were not carefully reading her messages. In addition to improving capability by clarifying study processes and observation tool content, the external facilitator was mindful about nurturing a two-way relationship and being supportive to improve motivation. For example, she followed up after the mid-implementation site visit to provide the internal facilitators with the positive feedback the site visitors had received from frontline staff participants.

We initiated phone calls with facility leadership over the course of the study to reinvigorate study participation. We also encouraged the study point of contact to review the audit and feedback reports with internal facilitators prior to their bright spots frontline huddles. Despite E-mailed reminders from the external facilitator, these meetings between nurse leaders and internal facilitators did not materialize.

In addition to direct facilitation, we tried to improve capability and motivation by cultivating a learning community midway through the study period by sharing successful strategies employed during intervention implementation at the second site (CLC B), including a system for tracking and training new frontline staff to participate in the intervention. The external facilitator reported, however, “by that time we had kind of lost [CLC A]; motivation was gone.”

Refining in CLC B

Our partners in CLC A required ongoing support from the research team to implement the LOCK-based intervention in their facility.

[There was] much more enthusiasm [in CLC B] than in [CLC A], thus we have done less “hand-holding,” that is, had less follow-up calls with the [CLC B] team. (Researcher)

We implemented new facilitation practices, however, to influence study participants’ capability and motivation. To maintain momentum of study-related activities and ensure continuity of communication in the event of staff-turnover, the external facilitator regularly inquired about upcoming absences among CLC partners. When absences of key individuals were expected to exceed one week—as in when the internal facilitator in one of the two participating neighborhoods transferred to another VA medical center—the external facilitator worked with the study points of contact to identify and engage an alternate internal facilitator.

Discussion and Implications

Our early experiences in implementing the LOCK-based intervention demonstrate the value of performing incremental implementation paired with ongoing formative evaluation when conducting nursing home research and QI activities. After initiating the research effort in a single site (CLC A), we periodically evaluated the progress of implementation efforts and adapted our implementation strategies—and even parts of the intervention—to enhance capability, opportunity, and motivation among staff members. Owing to ongoing communication with study points of contact and internal facilitators, we responded to all feedback that would enhance intervention implementation at that site. We learned to prioritize relationship-building over simply training our CLC partners on the intervention components and to use ongoing, personal communication to set joint research priorities. Experience also taught us to perform the training over a longer period to avoid overwhelming study partners who were busy with other administrative and clinical duties. Once the intervention began, our strategies to train frontline staff had to evolve to be more personal and systematized. Similarly, we needed to be flexible with the intervention and its implementation. Our team identified which elements of the LOCK-based intervention we believed to be integral that we would not change and creatively searched for those elements we believed to be peripheral that we would modify to make it easier for frontline staff to participate (Table 4). Bright spots frontline huddles were a core—and therefore unchangeable—component of the LOCK-based intervention. But we did develop new tools like easy-to-understand data reports and communication scripts to make huddles more meaningful and likely to occur.

Table 4.

Core Intervention Components and Adaptable Elements of the LOCK Intervention, Organized by Capability, Opportunity, Motivation, and Behavior (COM-B) Domain and LOCK Element

COM-B domain	LOCK elements	Core intervention components	Adaptable intervention components
Capability	Observe Collaborate in huddles	• Engaging and training internal facilitators • Nurse managers as initial huddle facilitators • Training staff to use observation tools	• Content of training • Training delivery method
Opportunity	Observe Facilitation	• Obtain support from local leadership • Engage nursing staff at all levels • Use standardized observation tools	• Design of the observation tools (e.g., formatting, type of data collected) • Timing of intervention activities • Staff involved in the intervention (e.g., number, disciplines, units, shifts)
Motivation	Learn from the “bright spots” Keep it bite-sized	• Keep all activities “bite-sized” • Conduct huddles focused on bright spots • Audit and feedback of data	• Goal of QI project • Design of the feedback reports (e.g., type of data reported, formatting, analyses)

Note: LOCK = Learn from the bright spots, Observe, Collaborate in huddles, Keep it bite-sized; QI = quality improvement.

Despite demonstrated efficacy or benefit to nursing home residents (Cranley et al., 2018), nursing home-based research is often poorly sustained after study completion, especially when external researchers have primary implementation responsibility. The researcher-as-implementor may solve known challenges such as staff members’ poor adherence to intervention and data collection protocols (Maas et al., 2002). Lack of engaged stakeholders, however, may preclude intervention uptake, scale-up, and spread under real-world conditions. Facilitation, grounded in organizational learning theory (Argote & Miron-Spektor, 2011), is a powerful approach to sustain evidence-based practices and innovations in complex clinical programs (Baskerville, Liddy, & Hogg, 2012). As we detail elsewhere (Pimentel et al., 2019), facilitation was critical to equipping internal facilitators with implementation skills and tools to sustain the intervention.

Formative evaluation such as ours may advance the implementation science relevant to nursing homes and remedy the lack of robust evidence to inform the successful dissemination and implementation of evidence-based practices in these settings (Lourida et al., 2017). The integration of research and implementation experience is necessary if we are to improve implementation processes and, ultimately, health outcomes (Estabrooks, Brownson, & Pronk, 2018). One way to do this is to systematically identify potential barriers and facilitators and plan for multifaceted strategies that address the myriad needs and desires of relevant stakeholders (Lourida et al., 2017). We used implementation- and progress-focused formative evaluation to assess how well we researchers supported our CLC partners as they implemented the intervention during the formal study period. Based on our understanding of barriers and facilitators to implementation derived from interviews with CLC study participants (Mills et al., 2019), we made changes to our approach in each participating site to improve staff participation and achieve the best possible implementation outcomes. As in other efforts to achieve small-scale implementation in nursing homes (Malterud, Aamland, & Iden, 2018), formative evaluation enabled us to improve our readiness as researchers to support the research effort, specifically through real-time feedback on the quality of our interactions with our partners during the preimplementation and implementation processes. Had we performed wholesale implementation in all six sites without clear understanding of these considerations, our study would likely have failed.

Limitations of our study should be noted. Owing to resource and time constraints, we did not perform formal implementation fidelity assessments to determine the extent to which the discrete implementation activities influenced uptake of the LOCK-based intervention. We continuously tailored our approach for each CLC based on trial-and-error and what we observed to work (or not). We therefore report only on the impact of this “bundled” approach. We also did not ask external and internal facilitators to track their activities in a time motion tracking log, as others have proposed for reporting on the breadth and depth of successful facilitation (Owen et al., 2019). We believe that such a request would have significantly impacted which CLCs were willing to participate, with only the most organized and energetic of potential internal facilitators agreeing, thus reducing the generalizability of our sample. Instead, the external facilitator summarized her key facilitation activities and weekly debriefings with her CLC counterparts and then shared her observations at weekly research team meetings. These limitations notwithstanding, our findings carry implications for the growing evidence base on what it takes to successfully initiate and sustain research and QI in a nursing home setting.

This formative evaluation reveals important lessons for researchers or clinical practitioners who are interested in implementing research programs or QI efforts in nursing homes. The challenges of performing research and QI in nursing homes, as described above, require meaningful collaboration between researchers and practitioners and between leadership and frontline staff. Successful implementation requires being flexible in one’s approach to implementing new practices. It also takes time, because trust is gained slowly through frequent, sustained, and genuine interactions. It is critical to cultivate social relationships and teamwork—both between researcher and staff and among staff themselves—that then serve as the foundation for this type of work. The methods outlined here provide a draft blueprint for how researchers’ strategies can evolve over time in response to challenges that are likely to emerge during the implementation of a research intervention in the nursing home setting.

Funding

This work was supported by the Department of Veterans Affairs, Veterans Health Administration, Office of Research and Development, Health Services Research and Development [I01HX000797] and Department of Veterans Affairs, Veterans Health Administration, Office of Research and Development, Rehabilitation Research and Development [IK2RX001241 to W. L. Mills].

Acknowledgments

The authors thank Dr Rebecca S. Allen (University of Alabama, Tuscaloosa) and Dr Kristen Dillon (Edith Nourse Rogers Memorial Veterans Hospital) for significant contributions to data collection.

Conflict of Interest

None reported.