Implementing organizational physical activity and healthy eating strategies on paid time: process evaluation of the UCLA WORKING pilot study

Abstract

Integrating organizationally targeted wellness strategies into the routine conduct of business has shown promise in engaging captive audiences at highest risk of obesity and obesity-related health consequences. This paper presents a process evaluation of the implementation of the University of California, Los Angeles, Working Out Regularly Keeps Individuals Nurtured and Going (WORKING) pilot study. WORKING focuses on integrating physical activity and nutrition practices into workplace routine during non-discretionary paid work time. The purpose of the evaluation was to assess the quality of implementation and to understand factors that facilitated or hindered organizations’ full uptake of the intervention. Fifteen worksites were randomly assigned to an intervention condition. Qualitative data were gathered through routine site visits and informant interviews conducted throughout each worksite’s intervention period. Worksites were classified into one of four implementation success categories based on their level of adoption and maintenance of core intervention strategies. Six key factors emerged that were related to implementation success: site layout and social climate, wellness infrastructure, number and influence of Program Champions, leadership involvement, site innovation and creativity. This pilot study has informed the conduct of WORKING II; a cluster randomized controlled trial aimed at enrolling 60–70 worksites in Los Angeles County.

Introduction

Obesity and its co-morbidities impose a disproportionate burden on low income, US born ethnic minority communities [1]. Rates of obesity, and obesity’s behavioral underpinnings, physical inactivity and unhealthful eating, are significantly higher among ethnic minorities, even after taking into account socio-economic status [2–5]. Despite this reality, research aimed at reducing obesity and improving physical activity (PA) and eating patterns in underserved communities has been limited [6]. Furthermore, most obesity control efforts have been targeted to individuals, with limited engagement, effectiveness and sustainability, particularly among ethnic minority populations [7–9].

There is a growing need to investigate interventions: (i) implemented in diverse organizational settings where ‘captive audiences’ already spend much of their time and (ii) promoting ‘push’ policies and practices at the organizational level. Push strategies are designed to modify culture and environment to make healthier activities the ‘default’ or easier options that one must opt out of to avoid (e.g. PA breaks during meetings), while making less healthy activities more difficult (e.g. prolonged sitting). This is in contrast to the ‘pull’ strategies traditionally employed in past worksite-based PA and nutrition interventions that rely on individual motivation (e.g. offering discounted gym memberships on non-paid time) [10, 11]. The corresponding language in behavioral economics for push strategies is ‘nudge’ strategies [12]. The success with which this program engages workers will depend on the degree to which the program is implemented as an opt out rather than an opt in strategy. Behavioral economics has shown that ‘choice architecture’ can nudge individuals to make healthier choices even while preserving their freedom to make less healthful choices [12]. Similarly, employers can mandate that work meetings that last an hour or longer include a 10-min ‘Instant Recess®’ (IR) PA break, but meeting participants are free to leave the meeting during this 10-min break if they wish—the opportunity is mandatory but participation is voluntary. In our experience, such ‘choice architecture’ results in 90+% participation [13].

Organizational push strategies have shown promise in engaging ‘hard to reach’ populations at highest risk of sedentariness and poor nutrition [13–17]. However, detailed process evaluations of these interventions are rarely published, making variability in outcomes difficult to interpret [8, 9]. According to Rogers’ Diffusion of Innovations theory, the success of implementation depends on factors at the organization level, characteristics of change agents within the organization, the nature of the ‘innovation’ and the larger context in which organizations are nested [18]. Factors at the individual and organizational levels within these settings, such as readiness for change, lack of resources, competing demands and incongruence with standing organizational priorities have been cited as barriers to effective adoption, implementation, maintenance and evaluation of culture change-oriented interventions [19–23]. Efforts must be made to better understand and account for organizational context when intervening in dynamic ‘real-world’ settings where project goals and objectives may not be given top priority or adequate attention.

This paper presents a qualitative evaluation of the implementation process of the University of California, Los Angeles (UCLA) ‘Working Out Regularly Keeps Individuals Nurtured and Going’ (WORKING) pilot study. The WORKING intervention strategy focused on integrating PA and healthy eating policies and practices (e.g. exercise breaks, healthy food options at meetings) into workplace routine during non-discretionary paid work time. The purpose of this paper is to assess the success of intervention implementation among organizations randomized to an intervention condition, and to understand factors that facilitated or inhibited implementation throughout the study period.

Materials and methods

The WORKING pilot intervention trial engaged 25 health and human service organizations (private and public), over a 3-year period. These organizations were selected because their employees consisted primarily of ethnic minority women (e.g. African American, Latina). These women are a priority population for health promotion [13] and targeting this group for such efforts may lead to benefits beyond those obtained through individual or organizational level efforts, given their roles as family and community gate keepers [24]. Participating sites were randomized to one of three study conditions: comprehensive intervention, standard intervention or wait-list control. The study protocol was approved by the institutional review boards of UCLA and the Los Angeles County Department of Public Health.

Development of the intervention

One year of formative research activities preceded recruitment of worksites for this intervention study. A series of 22 community dialogs were held, soliciting input from a total of 188 individuals drawn from our target population (e.g. employees representing 59 health and human services organizations) to identify PA and healthy eating strategies that would be appropriate and feasible for worksite settings. A 12-member Community Investigator Team was then assembled to provide additional guidance in refining intervention strategies.

The resulting intervention consisted of (i) compulsory ‘core’ elements considered to have the greatest feasibility and cultural congruence, based on relevant research evidence and our prior experience [13, 15, 16, 24] and (ii) optional ‘elective’ elements to provide flexibility and a ‘menu of options’ consistent with organizational needs identified in previous studies engaging ethnic minorities [24, 25] (see Table I).

Table I.

WORKING intervention policies and practices

Core (‘push’ strategies required for participation)	Elective (‘menu of options’)
PA related
Incorporate 10-min exercise breaks during lengthy meetings and at a certain time of the day, on paid time. (’Lift Offs!’ or IR™ breaks)	Encourage more casual attire compatible with lifestyle integration of PA
Support other individual and group exercise during the routine ‘conduct of business’, (e.g. walking meetings and scheduling sit-down meetings in rooms at a short distance from attendees’ workspace)	Change organizational culture to promote and reward lifestyle activity, e.g. standing up at intervals, doing airline exercises in one’s chair, stretching during meetings
Post stair prompts and ask managers to take the lead in using stairs instead of elevators	Provide substantive incentives for mass transit use
Food and nutrition related
Provide a bowl of fresh fruit and healthy snacks in the reception or central congregating area	Install water fountains, coolers or dispensers
Include healthy food choices at meetings in which refreshments are served	Include at least 50% healthy and competitively priced food choices in workplace vending machines, cafeterias and on-site food vendor offerings
Establish healthy food procurement and fundraising policies for catering and conference/meeting facility menus	Include language in subcontracts mandating or providing incentives for suppliers’ adoption of healthy/fit practices and policies
Replace candy/cookie jars on organizational leaders’ desks with bowls of fruit or small packages of nuts (preferably unsalted) or dried fruit/nut mix	Place calorie information about products in workplace vending machines
Broad policy adoption
Adopt formal written policies institutionalizing wellness practices and informal policies	Include wellness policy implementation duties in job description of senior manager and line or administrative staff, i.e. organize movement breaks, ensure food procurement policy adherence

Following the formative research, worksites were recruited for the WORKING pilot study through a rolling enrollment process over the final 2 years of the study period. Sites were grouped into three cohorts based on their date of recruitment and randomly assigned to three treatment groups: comprehensive intervention, standard intervention or no intervention (wait-listed control).

Worksites randomized to comprehensive and standard interventions implemented intervention activities for a period of 6 months. Sites randomized to the comprehensive group received training and resources for implementing core and elective elements related to PA and healthy eating. Standard sites received training and resources for implementing only the PA-related elements. Control sites did not receive intervention resources until the completion of the 6-month intervention period for their respective cohort. Our team made special efforts to support and evaluate the integration of IR breaks into workplace routine—brief 10-min peer-led group exercise breaks consisting of low-impact calisthenics and/or simple dance movements set to upbeat rhythmic music [17]. Sites were encouraged to incorporate IR breaks versus longer more intensive forms of exercise (e.g. Zumba, high-impact aerobics) to maximize employee participation and operate within the constraints of company time allotted for PA. Sites were free to explore other forms of PA as part of their ‘elective’ strategies; however, these activities were not independently evaluated.

Process of intervention implementation

Following site randomization, peer leaders or ‘Program Champions’ (PCs) were recruited from each intervention site. PCs played a crucial role in implementing intervention activities at their worksite, communicating with site leadership regarding project activities and serving as liaisons to WORKING Project staff by providing regular updates on intervention implementation. PCs were identified through self-referral or co-worker recommendation. We envisioned the ideal PCs to be: (i) fitness enthusiasts comfortable in leading group exercise and (ii) opinion leaders with social capital within the organization, preferably with long tenures at the site. These traits characterize those of effective ‘change agents’ who are best able to facilitate the uptake of innovations and culture-change policies within organizations [18, 26, 27]. Given the difficulty of identifying individuals who possessed both of these qualities, a mixture of complementary traits was sought among the employees recruited from each site. We encouraged sites to recruit PCs from different work subunits to distribute intervention tasks and maximize the reach of the intervention to all staff employees; however, we deferred to organizations when it came to making final decisions about the selection of PCs.

Employees recruited as PCs participated in a half-day interactive group training session, adapted from previous activity break promotion efforts [13]. The core training curriculum consisted of a background and rationale for the study; a step-by-step guide on implementing, sustaining and troubleshooting WORKING strategies at the worksite and an interactive PA leader trainer where attendees demonstrated their proficiency in leading IR breaks. Additionally, PCs from worksites assigned to the comprehensive condition received training focused on integrating healthy eating strategies into the workplace. At the completion of the training, PCs received toolkits containing additional learning aids, PA CDs and DVDs and an assortment of branded incentive items (e.g. pedometers, lunch bags, water bottles) to be distributed to employees at their respective worksites.

Project staff visited sites 1–2 weeks following the training to ensure that an implementation strategy had been created and that resources were in place to initiate the intervention. PCs were instructed to officially initiate or ‘kick-off’ intervention activities no later than 2 weeks following the PC training. Following intervention kick-off, project staff met regularly with intervention sites (every 4–6 weeks) throughout the study period to provide ongoing technical assistance to PCs and to evaluate the degree of intervention implementation. During site visits, project staff completed environmental audits that were used to monitor changes in the physical and social environment of the worksite (i.e. stair prompts, snack basket, etc.) [24]. Site visits also provided an opportunity for project staff to collaboratively brainstorm with PCs, key contacts and organizational leaders to identify emerging challenges and develop solutions (e.g. strategies to stimulate staff engagement, recruitment of additional PCs, etc.) as needed. At the conclusion of each worksite’s 6-month intervention period, project staff hosted a ‘wrap-up session’ with PCs and site leadership. Each site was provided a report that summarized the environmental audit and process data gathered by project staff throughout the intervention period and included a list of recommendations for further developing wellness activities.

Data collection and analysis

Process evaluation data were collected through environmental audits by project staff and through site visits to organizations where interviews were conducted with PCs. Occasionally, worksite key contacts and leadership representatives were queried about additional details to support evaluation efforts.

Observable changes to the physical and social environment of each worksite were assessed through formal environmental audits, and observations made during routine site visits. The environmental audit tool used in this study was a simple printed document, adapted from our previous research efforts [24]. The audit tool was administered by research team members to assess key physical and social environment aspects relevant to healthy eating and PA: physical layout of the worksite; availability of outdoor spaces for walking or PA, presence of cafeterias, kitchens and employee break areas; presence and content of snack baskets, vending machines and other food sources readily accessible to staff; availability of water coolers and fountains, presence of stairs and posted stair prompts and visibility of promotional materials encouraging healthy eating and PA. PA breaks that occurred during the visit were also documented. Typically conducted by two research team members, the environmental audit tool was completed by each team member present at the time of the audit and then consolidated into one data report. Any disagreement between team members on environmental audits was resolved through discussion following the site visit. All intervention sites received an environmental audit prior to intervention implementation (baseline), and attempts were made to conduct follow-up audits by the end of the intervention period. The information drawn from formal environmental audits was supplemented by environmental observations made by team members during regular site visits that occurred every 4–6 weeks throughout the project period.

Via conversations with PCs, site visits focused on assessing the level of implementation of core elements of the intervention (e.g. daily PA breaks, PA breaks at meeting, upkeep of snack baskets and other healthy food options, etc.), identifying any additional components implemented and discussing and brainstorming solutions for challenges. Site visits were scheduled every 4–6 weeks throughout each site’s project period. PCs were also asked to discuss their interactions with leadership and key decision-makers, lapses in intervention activities and organizational events that might have impacted implementation of the intervention (e.g. staff layoffs, changes in leadership, major priority shifts for the organization, etc.). The timing of site visits was scheduled to accommodate the availability of all PCs; therefore, nearly, all PCs from a particular site attended each site visit. During the visits, research team members encouraged attendees to speak candidly, and input from each was solicited about their experiences and insights about the process of implementation, challenges and any need for technical assistance. PCs from a particular site were typically in agreement about their perceptions regarding intervention implementation, and a consensus was often reached spontaneously. A simple printed form (e.g. ‘site visit notes’) was created to capture data collected during site visits.

Data from environmental audits and site visit notes were used to develop an organizational profile for each site. These profiles summarized more concisely the large amount of data collected from each site throughout the intervention and to facilitate comparisons across intervention sites. The organizational profiles contained (i) background information and demographics for each site (e.g. staff size), (ii) a summary of major organizational-level changes that occurred during the intervention (e.g. layoffs, changes in leadership), (iii) summary of key physical environment observations from audits and site visit notes, (iv) summary of initial implementation, kick-off’ activities, (v) a chronological summary of implementation for the core strategies (e.g. frequency of PA breaks, availability of healthy food) across the course of the intervention period, (vi) description of ‘elective’ strategies implemented and any site-specific innovations and (vii) persistent challenges and solutions implemented.

This information was used to assign each site to one of four categories reflecting the level of success of implementation: ‘model adopters’, ‘fair adopters’, ‘poor adopters’ and ‘wipeouts’. These categories reflect three aspects of implementation: (i) degree of implementation of the core intervention strategies (i.e. most, few, none), (ii) presence or absence of lapses in implementation activities (e.g. many, few or none) and (iii) whether or not activities were sustained until the end of the intervention period (e.g. sustained and not sustained).

Following the assignment of sites to success categories, organizational profiles were reviewed and several were selected to build descriptive case studies illustrating the most and least successful worksites. Common themes and factors that appeared to be associated with implementation success emerged in the process of developing the case studies. Organizational profiles were again reviewed to identify additional factors related to implementation success. In total, six factors were identified (see Table II). Sites were then rated by a project staff member and study investigator on each of the six factors using organizational profiles and data from field notes. For each of the six factors identified, sites were assigned a rating to indicate whether each factor was clearly present (+), clearly absent (−) or questionable (+/−). See Table II for descriptive information about the success factors identified and the factor rating scale.

Table II.

Organizational characteristics and intervention factors related to implementation success

					Organization and intervention factorsa
	Worksite type	Number of employees	Intervention group	Implementation successb category	Layout and social climate	Wellness infrastructure	Employee to PC ratio	PC Impact	Leadership impact	Innovation and creativity
Model adopters	County public health clinic	80	Comprehensive	Model	+/−	+	+	+	+	+
	Community network for cancer screening	10	Standard	Model	+	−	+	+	+	+/−
	Health-related programming for families	100	Comprehensive	Model	+	+/−	−	+	+	+/−
‘Fair adopters’	Health-related programming for women	20	Standard	Fair	+	+/−	+	+/−	+	−
	Economic and social Justice advocacy organization—four sites	110	Standard	Fair	+/−	+/−	+/−	+/−	+/−	+
	Multipurpose senior center: multi sites	315	Comprehensive	Fair	+	+/−	+/−	+/−	−	+/−
	Administration unit	87	Standard	Fair	+	+/−	+/−	+/−	+/−	−
	Childcare center in public housing	32	Comprehensive	Fair	+	−	−	+/−	+/−	+/−
‘Poor adopters’	Vaccine-preventable disease services	96	Comprehensive	Poor	+/−	+	−	+/−	−	+
	Personnel unit	100	Comprehensive	Poor	−	+/−	+	+/−	−	+/−
	Community development agency	40	Standard	Poor	−	−	+	−	+	+/−
	County public health clinic	52	Standard	Poor	+/−	+/−	−	−	+/−	−
’Wipeouts’	County public health clinic	80	Standard	Wipeout	+/−	+	−	+/−	−	−
	Childcare advocacy organization	350	Comprehensive	Wipeout	−	+/−	−	−	+/−	+/−
	Parks and recreation unit	125	Comprehensive	Wipeout	+/−	−	−	+/−	+/−	+/−

a

Organizational and intervention factors associated with implementation success:

Layout and social climate:

• (+) = Organization centralized to one distinct physical location, staff organized in proximal or non-physically separated organizational units, with abundant social interaction among staff.

• (+/−) = Organization is centralized to one distinct location; staff is organized into phyisically separated units, with limited social interaction among peers.

• (−) = Organization is decentralized to several satellite sites; staff is physically separated, with minimal social interaction within and across sites.

Wellness infrastructure:

• (+) = Formally recognized wellness infrastructure (e.g. wellness programs and committees) or activities.

• (+/−) = Informal wellness activities, typically peer-mediated.

• (−) = No wellness activities held.

Employee to PC ratio:

• (+) = 10 employees to 1.

• (+/−) = 10–15 employees to 1.

• (−) = 15 employees or more to 1.

PC Impact:

• (+) = PCs adequately distributed across all organization units and locations, possess flexible job schedules and decisional latitude to accomplish intervention tasks, effective at delegating responsibilities to other employees.

• (+/−) = PCs adequately distributed across all organization units; possess some job flexibility and decisional latitude, some success at delegating responsibilities.

• (−) = PCs not adequately distributed across all organizational units, possess little job flexibility and decisional latitude, not successful at delegating responsibilities.

Leadership Impact:

• (+) = Leadership verbally endorses intervention activities, actively participates, and readily provides input and support for troubleshooting.

• (+/−) = Leadership verbally endorses intervention, participates occasionally, and provides some input and support for troubleshooting.

• (−) = Leadership gives few verbal endorsements of intervention, does not actively participate and provides little to no support for troubleshooting.

Innovation and creativity:

• (+) = Elective strategies employed in addition to sustaining core strategies.

• (+/−) = Elective strategies employed to augment or substitute core strategies.

• (−) = No elective strategies employed.

b

Implementation success categories:

• ‘Model adopters’: majority/all core elements implemented, sustained throughout intervention period, few or no lapses in activity.

• ‘Fair adopters’: some core elements implemented, elements sustained throughout intervention period, noticeable lapses in activity.

• ‘Poor adopters’: few to no core elements implemented, elements poorly sustained, large lapses in activity.

• ‘Wipeouts’: majority/all core elements implemented initially, no elements sustained, large lapses in activity.

Results

Of the 25 organizations recruited to the WORKING pilot study, 15 were randomly assigned to either the ‘comprehensive’ or ‘standard’ intervention conditions; the remaining 10 organizations were wait-list controls. Intervention sites included public Los Angeles County Department of Public Health work units as well as private non-profit health and human services organizations located throughout Los Angeles County. Organizations varied in size and physical layout, from large multi-location organizations (with up to 300 employees) to smaller single-site organizations (of 10–50 employees). Employees across all participating sites were predominantly female, middle-aged and ethnic minority (mostly African American and Latino). See Table II for information about intervention sites.

Evaluation of implementation success

Worksites were categorized into four levels of implementation success categories based on the extent to which they adopted and maintained the core elective elements throughout their intervention period. SeeTable II for more detailed descriptions of success categories. Three sites were identified as ‘model adopters’, having the greatest success at implementing and sustaining most or all core elements with few, if any, minor lapses in activities. Five sites were categorized as ‘fair adopters’, having success implementing and sustaining some core elements, but with noticeable lapses in activities. Four sites were categorized as ‘poor adopters’, having limited success with implementing or sustaining any elements and reporting major lapses in activities. Lastly, three sites were categorized as ‘wipeouts’. These sites had initial success with implementing most or all intervention elements; however, before the end of the intervention delivery period, nearly all intervention-related activities had ceased.

Organizational and intervention factors related to success

In addition to evaluating the degree of successful implementation, we were interested in whether or not organizational-level and intervention-related factors could be identified that were associated with success or failure of intervention implementation. After reviewing process data and organizational profiles, six common factors emerged (see Table II). Two of these factors involved pre-existing social and structural characteristics of sites: (i) worksite layout and social climate and (ii) wellness infrastructure. Related to worksite ‘layout and social climate’, sites with one physical location and common space where employees frequently interacted were more successful in implementing the intervention compared with sites with multiple physical locations and fewer opportunities for staff interaction. Sites that had either formal or informal wellness activities in place prior to the intervention were more likely to be successful compared with sites with no pre-existing wellness activities. The other four factors involved aspects of how sites carried out intervention activities. ‘Staff to PC ratio’ was important with more successful sites having a higher a concentration of PCs for a given staff size. We also found that the level of skill or ‘impact of the PCs’ to be influential; not so much related to the technical aspects of leading PA breaks but more to their ability to engage peers in intervention activities and communicate with leaders to troubleshoot challenges. The degree to which ‘leadership was engaged’ in the process of implementation was also related to success. Finally, we found that sites willing to tailor core intervention strategies (e.g. create their own PA routines) and initiate ‘innovative’ elective strategies were more likely to fully implement the intervention than sites that did not tailor strategies.

Case studies

The following case studies describe the process of intervention implementation at three participating sites and highlight factors related to implementation success and failure. We selected one site that was particularly successful (model adopter), one site that was initially successful but ceased most activities over time (wipeout) and a site that was unsuccessful in both implementing and sustaining program activities (poor adopter).

Local county public health clinic (Site A)

One of the most successful sites was a public health clinic operated by the Los Angeles County Department of Health Services. This site had a staff of 80 employees working within eight distinct work units and one physical location.

A total of eight employees were trained as PCs, including a top-ranking organizational leader and middle management decision-maker. An intervention kick-off event occurred two weeks after the PCs received training.

The site was particularly successful in holding IR PA breaks an average of 4–5 days per week. PCs initially had difficulty engaging clinical and administrative employees, as their work schedules were more unpredictable and dependent on patient load compared with other workers. An additional afternoon PA break was conducted to accommodate these employees. On average, 10–15 employees (15–22% of staff) participated in each activity break. PCs tailored their daily PA offerings by developing a unique routine that incorporated three popular culture line dances. PA breaks were integrated into the agenda of agency ‘all-staff’ meetings throughout the intervention period, yielding participation rates of 70–90% of attendees. PA breaks were also conducted during monthly public health nursing meetings.

Site A faced challenges in sustaining a consistent supply of healthy snacks. PCs attempted to revive a previously defunct ‘Snack Shack’ where food items were purchased in bulk and re-sold to individual employees at below-retail prices. However, this effort was abandoned because it was considered too labor intensive to maintain. Food was typically not served at meetings thus this was not an area of focus for the intervention.

Site A successfully integrated a number of innovative ‘elective’ strategies to bolster employees’ involvement in the core intervention activities. Visually appealing displays of PA and healthy eating promotional items were posted on several vacant bulletin boards throughout the facility and routinely updated throughout the intervention period. To improve employee engagement in the activities, PCs created a homegrown incentive-based wellness competition. Employees from each work unit formed teams and were awarded points for participating in PA breaks, leading PA breaks, logging pedometer readings and other activities. The work unit with the highest number of points received a small prize as well as public recognition. PCs also initiated a successful ‘Produce of the Month’ campaign, encouraging employees to share whole foods and recipes focused on the featured produce for that month.

Site A successfully implemented and sustained most of the core (and several elective) intervention elements through the coordinated efforts of PCs and a highly supportive organizational leader. PCs made a number of innovations in order to retain staff interest and responded effectively to challenges that surfaced during the intervention period.

Non-profit childcare advocacy organization (Site B)

The following describes a ‘wipeout’ site that initially succeeded in implementing the intervention but ultimately failed to sustain any intervention elements long term. This site was a private non-profit advocacy organization with 350 employees occupying three physical locations within a 10 mile radius. The majority of employees were housed within two large buildings located at the agency’s headquarters location, which is where the bulk of intervention implementation efforts were focused.

PC training took place at the headquarters site, engaging employees from all three organization locations. Six employees from the headquarters location were trained as PCs. A well-publicized day-long launch event was held at the headquarters site prior to the initiation of activities.

At the onset of the intervention period, Site B was quite successful at implementing PA breaks most days of the week, integrating PA breaks into regular meetings and sustaining healthy food items in snack baskets. In addition to core intervention activities stated above, Site B hosted monthly ‘healthy potlucks’ for all employees during the first 2 months of the intervention.

Midway through their intervention period, Site B encountered challenges that ultimately derailed the intervention. The decentralized nature of the organization made it difficult for project staff to provide the technical assistance and monitoring needed to ensure adequate implementation in the two headquarters buildings, much less at satellite sites. A major restructuring of the organization occurred during the intervention period, which shifted priorities away from sustaining intervention activities. Although the organization regained stability by the end of the intervention period, staff interest in group-oriented health promotion had waned. Furthermore, several employees initiated a ‘Biggest Loser’ weight-loss competition without consulting PCs or research team members. The competition, with its emphasis on individual weight-loss and financial incentives for winning, was popular among staff but undermined the organizational strategies promoted by WORKING. Despite significant support from top-level leadership, a well-attended launch event and initial success in implementation, Site B failed to sustain any of the intervention strategies throughout the intervention period.

Local County public health clinic (Site C)

Site C was classified as a ‘poor adopter’, encountering implementation challenges early in the project which were not overcome despite considerable troubleshooting efforts by both PCs and WORKING project staff. One employee, a recent recruit to the organization with considerable offsite field responsibilities, was recruited and trained as a PC. Despite several attempts, the PC was unsuccessful at recruiting other employees to assist with facilitating intervention activities.

PA breaks were initially conducted 1–2 days per week and were incorporated into monthly all-staff meetings. From the outset, the PC had great difficulties engaging clinical and front line staff in PA breaks, and few employees participated in PA breaks at meetings. The PC also encountered logistical challenges (e.g. no stereo equipment, problems securing space for activity), which further hindered the implementation of intervention activities.

Although they supported the project verbally, worksite leaders were slow (or failed) to actively assist in troubleshooting efforts. Ultimately, the responsibility of being solely responsible for facilitating intervention activities overwhelmed the PC, leading to several complete lapses in activity during the last 2 months of the intervention period. Implementation was further challenged by an individually focused Biggest Loser competition instigated by other employees. Research team members and the PC suggested joining forces with the competition organizers but were rebuffed, with the result that the competition undermined WORKING’s group-based strategies.

Discussion

The purpose of the WORKING pilot study was to develop and implement an organizational-level obesity control intervention, utilizing strategies to promote PA and healthy nutrition that were feasible in work settings and customizable for a particular organization. This process evaluation highlights key factors that may promote or hinder an organization’s ability to successfully implement and sustain PA and healthy nutrition strategies among sedentary employees in health and human service settings.

Agencies whose employees are located within a single central location may be better suited to implement organizational-level PA and nutrition strategies than organizations with multiple locations. Dissemination of new organizational practices may be facilitated in sites that are centralized. In contrast, sites where employees are separated structurally (e.g. satellite locations, private offices versus open cubicles) or functionally (e.g. front office staff versus clinical staff) may develop smaller ‘microcultures’ that are less amenable to broad organizational norm changes. In these settings, efforts must be taken to identify diverse microcultures’, assess their specific barriers and facilitators to adopting new practices and tailor strategies to facilitate their active engagement. There is another possible structural explanation—smaller organizations may also have less staff time or proportionally more demands on workers’ time that hinders implementation of wellness activities.

PCs played a vital role in implementing the intervention, including tailoring strategies to best fit the specific needs of the worksite, brainstorming new strategies, troubleshooting problems and working with leadership to ensure that wellness remained ‘on the agenda’ of the organization. PCs with more experience within an organization, a higher degree of decision-making power and flexible job responsibilities were more successful in carrying out program-related responsibilities compared with less experienced employees with more ‘time-sensitive’ job responsibilities (e.g. front desk staff) and less social capital. Employees with some degree of personal enthusiasm for fitness and active living tended to gravitate to PC roles within their organization. However, we found anecdotally that these characteristics were not sufficient for the success of PCs. It was equally or more important that PCs had the ability to make decisions on behalf of the organization, a willingness to engage leadership in project efforts and a job position that was flexible enough to accommodate the additional responsibilities of serving as a PC in addition to their day-to-day duties. The number of employees willing and ready to serve as PCs was also associated with successful implementation. Not surprisingly, sites with more PCs were able to carry out activities more consistently than sites with fewer champions. Sites with fewer PCs were more likely to experience ‘champion’s fatigue’, where PCs became too overwhelmed with balancing job responsibilities and the demands of facilitating wellness activities to continue implementing the intervention.

Based on our experience, leadership plays a crucial role in ensuring successful implementation. In addition to sanctioning wellness strategies at the onset of the intervention, leaders were instrumental in ensuring employees had equitable access to wellness activities and resources on paid time, in enforcing new wellness policies as part of the organization’s ‘standard conduct of business’, and assisting PCs with troubleshooting implementation issues. In our ‘model adopter’ worksites, organizational leaders remained closely involved in implementation activities throughout the intervention period and even regularly participated in activities with employees. Less successful sites tended to have limited or inconsistent involvement of leadership, especially when it came to troubleshooting challenges.

Our findings also highlighted the need to involve individuals at different levels of the organizational hierarchy including top-level organizational ‘leaders’ (e.g. directors, ‘C-suite’ executives) and middle management ‘key decision-makers’ (e.g. supervisors, coordinators, program managers). Initially, we focused our efforts too heavily on engaging top-level organizational leaders. Although ‘buy-in’ from top leaders was necessary, we soon realized that engaging middle managers was key to ensuring the execution of day-to-day program activities. It was particularly advantageous to enlist middle managers to serve as PCs. In several instances when we were not able to engage middle managers, we encountered employee resistance that derailed intervention efforts. Findings from this pilot study clearly indicate the need to establish a solid rapport with leadership and key decision makers from the outset and to maintain open lines of communication throughout the implementation process. These lessons learned have informed the conduct of the full-scale WORKING intervention trial; site leaders and decision-makers are now engaged in ‘Executive Leadership Orientations’ (ELOs) prior to initiating intervention activities. During these ELOs, leaders are oriented on the primary objectives of the intervention, offered guidance on how they as leaders can specifically support wellness activities and are given an opportunity to present any concerns they may have about fully supporting WORKING at their site. Following the ELO and implementation of intervention activities, leaders are further engaged in the process by receiving monthly feedback reports describing the status of activities at their site and are asked to invite project staff and PCs to executive-level meetings to discuss emerging challenges to the intervention.

A number of situational factors that influenced intervention implementation should be acknowledged. Conducting worksite-based wellness activities during an economic downturn is extremely challenging. On numerous occasions, our efforts to engage and retain organizations in wellness activities were complicated by worksite funding shortfalls, staff reductions and concerns about introducing new activities into an already strained work environment. Implementation also fluctuated on a seasonal basis, with frequent lapses occurring near the end of the fiscal year, during winter holidays, summer vacations and when ‘high priority’ work activities precluded the usual organizational routine (e.g. influenza, H1N1 flu seasons in public health clinics). Another situational factor that had a substantial impact on implementation success was the popularity of weight-loss competitions modeled after the reality television show, ‘The Biggest Loser’. The emphasis of these competitions on individual behavior change usually detracted from our organizational-level efforts to implement practice and policy-level strategies. Given the popularity of program such as The Biggest Loser, we believe that proactively encouraging organizations to consider a restructured version of such competitions, incentivizing group versus individual-level outcomes instead of discouraging them altogether, is a prudent strategy.

Some limitations of the present study should be noted. Data drawn from site visits were based on self-report of PCs within each site. Also, the strategy of encouraging sites to personalize and adapt intervention activities to best accommodate the dynamics of their organization may limit the generalizability of findings to other worksites. In this paper, we restricted our focus to exploring factors at the organizational level that were associated with success or failure of intervention implementation based on data collected through site visits and environmental audits. Because of pilot study resource constraints, we were unable to examine whether individual-level outcomes were associated with implementation success or failure. This is a major focus of our ongoing randomized trial evaluating the effect of a similar organizational-level intervention implemented within a larger number of worksites over a longer period than the pilot project. Lastly, due to the extended nature of the intervention period, sites may have been influenced by organizational, social and other relevant factors not accounted for in this study.

Conclusions

Results of this process evaluation are consistent with the literature regarding the dissemination and implementation of ‘innovations’ within organizational settings. Consistent with Rogers’ Diffusion of Innovations theory, the success of implementation depended on factors at the organization level, characteristics of change agents within the organization, the nature of the innovation and the larger context in which organizations were nested [18]. Greenhalgh et al. [26] conducted a systematic review of the literature to understand the process of diffusion of innovations within health service delivery organizations. Also consistent with Greenhalgh and associates, our evaluation study found support for the importance of system antecedents, organizational approach to implementation and the external context in determining the success of implementation [26].

A number of lessons learned from this implementation evaluation have informed the conduct of our larger trial, a 5-year, two-armed cluster-randomized trial targeting 60–70 health and human services organizations in Los Angeles County. Intervention implementation has been enhanced by our efforts to more aggressively monitor and troubleshoot intervention implementation progress, particularly at the beginning stages of implementation. Many of these lessons have been memorialized in an IR Tool Kit sponsored by KEEN, a Portland, Oregon-based outdoor footwear company and may hence be more fully and consistently shared with worksites as they are recruited and enrolled.

Data from this pilot suggest that capitalizing on the initial enthusiasm expressed by staff and leaders is important to do before organizational challenges and champion fatigue slow the momentum of implementation. This fact notwithstanding, there must be a balance between immediate success of implementation and long-term sustainability. Without sufficient implementation, we will be unable to determine whether the intervention is ineffective or whether problems with implementation weakened a positive effect of the intervention. However, designing a too tightly controlled trial will reduce the chances that activities can be sustained long term. Care must be taken to account for the standing priorities and organizational climate of participating worksites, determine facilitators and barriers to effective implementation as early as possible and tailor intervention activities accordingly to maximize the likelihood for full implementation and sustainability.

Funding

This work was supported by a grant from the National Center on Minority Health and Health Disparities at the National Institutes of Health titled ‘Joining Forces with a Key Community to Reduce Obesity Disparities’ (1R24MD001762-01: PI: A.Y.). In addition, Mr. H.J. was supported in part through a fellowship award from the Jonsson Cancer Center Foundation under the mentorship of Dr. A.Y.

We acknowledge all WORKING Project staff, investigators and volunteers who have contributed to our efforts throughout the development and execution of this project. Additionally, we are grateful to the Los Angeles County Department of Public Health as our community collaborators for WORKING. Lastly, we extend thanks to our Comm-unity Investigator Team, PCs, partner organizations and participants for your patience and dedication throughout this process.