Abstract

Background: Social and physical environments are important drivers of socioeconomic inequalities in health behaviour. Although many interventions aiming to improve such environments are being implemented in underprivileged neighbourhoods, implementation processes are rarely studied. Acquiring insight into successful implementation may improve future interventions. The present study aimed to investigate factors influencing the reach, effectiveness, adoption, implementation and maintenance (RE-AIM) of social and physical environmental interventions aimed at promoting healthy behaviour in underprivileged neighbourhoods in The Netherlands. Methods: A large set of theory-based factors of successful implementation was assessed for 18 implemented interventions in three underprivileged neighbourhoods. Expert and target group panels scored the RE-AIM dimensions for each intervention. We analyzed the statistical significance of associations between theory-based factors and the actual RE-AIM in a statistical model, to identify factors associated with increased RE-AIM. Results: Six factors were identified: effectiveness and implementation success were higher when the target group was involved in the planning process, whereas maintenance increased in the absence of competition with other projects. If the current situation was inventoried during intervention development, the effectiveness, adoption and implementation were higher. These dimensions were also higher when the target group was informed before implementation. Involvement of the target group during implementation resulted in higher reach, effectiveness and adoption. Finally, lack of intervention staff worsened the reach. Discussion: This study contributes to the evidence base for effective implementation of environmental measures aimed at promoting healthy behaviours. In particular, interventions in which the target group was involved in the implementation process were associated with higher RE-AIM outcomes.

Introduction

Several studies have shown that unhealthy behaviours are more prevalent in low-socioeconomic status (SES) populations. People from lower-SES groups are, on average, less physically active, engage more in sedentary activities, eat fewer fruits and vegetables and are more likely to be overweight and obese than people from higher-SES backgrounds.1,2 Consequently, these individuals are less healthy and exhibit higher rates of morbidity and mortality.3–5

It has been posited that the social and physical environments in underprivileged neighbourhoods contribute to the disparities in health behaviours.6–10 Underprivileged neighbourhoods may offer less support and fewer opportunities for healthy behaviour because the environment is less attractive and less safe (more criminality and traffic) and there are fewer locations in which to exercise (such as parks, sports grounds and sports clubs).10,11 In addition, the social environment may contribute to health behaviour disparities between SES groups. For instance, it has been suggested that lower peer support, lower social participation and a smaller social network are negatively associated with health behaviours and are more prevalent among those who are socioeconomically disadvantaged.6–8

Social and physical environmental interventions aiming to improve health behaviours have been implemented in several countries in Europe, including The Netherlands. The most comprehensive example of such environmental interventions in The Netherlands is the governmental effort to support a selection of underprivileged neighbourhoods by addressing local problems relating to employment, schooling and education, housing, safety and social integration. This 10-year district approach was started in 2007 and was supported by a multi-billion Euro budget. The former Minister for Housing, Communities and Integration selected 40 of the most disadvantaged neighbourhoods, characterized by above-average unemployment rates, a high share of low-income households and criminality. These 40 neighbourhoods received increased financial aid and many interventions aimed at improving the social and physical environment were initiated. For example, projects were initiated to increase neighbourhood safety and home renovation, but also interventions directly aimed at promoting health behaviours (e.g. developing exercise areas for seniors, etc.).

Originally, health issues were not explicitly addressed in the district approach. However, mediation of the Ministry of Health, Welfare and Sports has resulted in the added ambition to improve the health of the district residents through an integrated approach focusing on healthy residents, healthy environment and a comprehensive prevention-oriented primary care. This theme was called ‘the healthy neighbourhood experiment’. Although these projects may have the potential to improve environmental conditions and decrease health disparities, earlier experiences show that many projects often fall short somewhere in the implementation process.12,13

A broadly used evaluation framework including multiple process indicators has been developed for the assessment of interventions beyond effectiveness. This so-called reach, effectiveness, adoption, implementation and maintenance (RE-AIM) framework was designed by Dr. Russ Glasgow and his associates to structure the assessment of a broad range of interventions and their public health impact.14 Important outcomes of efforts to translate project plans into practice are captured in five dimensions: the Reach into the target population; the Effectiveness or impact of an intervention on important outcomes; the Adoption in local settings, institutions and by staff; Implementation in the local setting (including consistency and cost) and Maintenance of interventions (and intervention effects in individuals) over time. The framework may particularity be useful in evaluating social and environmental interventions, as it addresses impact at the individual level as well as at the organizational level. Reach and efficacy are individual levels of impact, whereas adoption and implementation are organizational levels of impact. Maintenance can be both an individual and an organizational level of impact. It is pertinent to evaluate both levels because each provides valuable independent information of intervention impact.14 It is necessary to gain insight into the factors that affect the different RE-AIM dimensions, so that a better understanding of these determinants can increase the impact of future interventions. The aim of this study was to explore which factors are associated with the five dimensions of the RE-AIM framework in the implementation of environmental interventions in underprivileged neighbourhoods.14

Methods

The study methods have been reported in detail elsewhere.15 The stepwise approach to this methodology is outlined as follows:

Step 1: Selection of underprivileged neighbourhoods for this study

Thirteen of the 18 cities in which the 40 disadvantaged neighbourhoods were located took part in the ‘healthy neighbourhood experiment’ and received increased support for the implementation of interventions aimed specifically at reducing health disparities, e.g. initiatives that stimulate healthy behaviours. Within three of these 13 cities, one neighbourhood was selected for this study based on diversity of ethnic backgrounds of the residents, type of housing and pragmatic criteria: Eindhoven Doornakkers, Zaandam Poelenburg and Utrecht Overvecht.

Step 2: Selection of implemented interventions

Criteria for the inclusion of projects in this study specified that each had to (i) include measures aimed at improving the social and/or physical environment; (ii) target health behaviours such as physical activity, sedentary behaviour and/or dietary behaviour (as a primary goal or indirectly); (iii) be implemented after 2007 and (iv) be focused on adults.

Six different (but sometimes inter-related) interventions per neighbourhood were selected, which totalled 18 interventions (table 1). The interventions were selected using emergent purposive sampling.16 This sampling method allowed the authors to make sampling decisions during the process of collecting data and to ensure a heterogenic sample in terms of the intervention’s target behaviour, target population, type (physical, social or a combination of both), complexity and integration with other interventions. A more extensive description of the interventions that were included can be found in Supplementary Appendix 1.

Table 1

Selected social and/or physical environmental interventions in this study

Neighbourhood Intervention studied 
Eindhoven–Doornakkers 1. Walking club for seniors 
2. Exercise project 
3. Reconstruction Jan Toorop Street 
4. Gymnastics for parent and child 
5. Tasting gardens 
6. Front yard project 
Utrecht–Overvecht 7. Redevelopment ‘Klopvaartpark’ 
8. Working on a healthy weight 
9. Cycling lessons for immigrants by seniorsa 
10. Renovation courtyards of apartment blocks 
11. ‘Walk-in apartment’ in a block of flats 
12. Exercise mediator 
Zaandam–Poelenburg 13. ‘Liveability officials’ 
14. Exercise areas for seniors 
15. Gathering of multicultural women 
16. Exercise on prescription 
17. Exercise group for multicultural women 
18. Advisor of liveability 
Neighbourhood Intervention studied 
Eindhoven–Doornakkers 1. Walking club for seniors 
2. Exercise project 
3. Reconstruction Jan Toorop Street 
4. Gymnastics for parent and child 
5. Tasting gardens 
6. Front yard project 
Utrecht–Overvecht 7. Redevelopment ‘Klopvaartpark’ 
8. Working on a healthy weight 
9. Cycling lessons for immigrants by seniorsa 
10. Renovation courtyards of apartment blocks 
11. ‘Walk-in apartment’ in a block of flats 
12. Exercise mediator 
Zaandam–Poelenburg 13. ‘Liveability officials’ 
14. Exercise areas for seniors 
15. Gathering of multicultural women 
16. Exercise on prescription 
17. Exercise group for multicultural women 
18. Advisor of liveability 

a: This intervention was excluded because not enough data could be obtained from intervention staff.

Step 3: Assessment of potentially important factors

Factors that potentially affect RE-AIM dimensions were derived from implementation models developed by Grol and Wensing and Fleuren et al.15,17,18 (table 2). These ranged from characteristics of the intervention to characteristics of the organization, the coordinator and/or the project manager. The interviews were tape-recorded. Then, based on the recorded information, two researchers independently scored each intervention. For every factor that potentially affects RE-AIM dimensions, it was scored whether it was addressed. When the scores of the two researchers were dissimilar, consensus was reached on after discussion.

Table 2

List of potentially important factors that are associated with (un)successful implementation and use of environmental interventions to promote healthy behaviour in low-SES neighbourhoods, based on the Grol and Wensing17 model and others15,18

Characteristics of the intervention 
    Planning and preparation 
        1. Use of a theory in the planning process 
        2. Involvement of target group during process planning 
        3. Focus of the intervention on problems (target group/setting) 
        4. Use of knowledge of comparable interventions 
    The development of the intervention proposal 
        5. Specific (concrete) 
        6. Possibility for adjustments 
        7. Development of an action plan or protocol 
        8. Involvement of the target group during development 
        9. Problem analysis carried out 
        10. Degree of integration of the intervention 
        11. Inventory carried out during development 
        12. Involvement of intermediates during development 
    Analysis of current situation 
        13. Change complies with target group needs 
        14. Competition with other (local) projects 
        15. Embed plan in existing activities 
    Development and selection of strategy for implementation and dissemination of the intervention 
        16. Plan is attractive for target group (pr) 
        17. Communication to target group 
    Development, testing, implementation 
        18. Has the implementation strategy been tested? 
        19. Developed on account of a problem instead of available financing 
        20. Monitoring of the implementation process 
        21. Adequate budget for implementation 
        22. Compensation/allowance performers 
        23. Change causes (temporary) inconvenience 
        24. Visible results 
        25. Involvement of the target group duringimplementation 
        26. Actual frequency of use compared to intended use 
        27. Time-bound implementation and planning 
        28. Time available for implementation 
    Evaluation 
        29. Process evaluation carried out? 
        30. Ability to customize change 
        31. Is the long term taken into account? 
Characteristics of the organization 
        32. Organization size (large, medium size, small) 
        33. Is there a clear direction? (leadership, budget) 
        34. Lack of staff 
        35. Collaboration with professionals in the neighbourhood 
        36. Previously existing local cooperation 
        37. Available expertise in the organization in relation to the intervention 
        38. Decision-making process/procedures in the organization (top-down/ bottom-up) 
Characteristics of the coordinator/project manager 
        39. Enthusiasm 
        40. Expertise related to the intervention 
Characteristics of the intervention 
    Planning and preparation 
        1. Use of a theory in the planning process 
        2. Involvement of target group during process planning 
        3. Focus of the intervention on problems (target group/setting) 
        4. Use of knowledge of comparable interventions 
    The development of the intervention proposal 
        5. Specific (concrete) 
        6. Possibility for adjustments 
        7. Development of an action plan or protocol 
        8. Involvement of the target group during development 
        9. Problem analysis carried out 
        10. Degree of integration of the intervention 
        11. Inventory carried out during development 
        12. Involvement of intermediates during development 
    Analysis of current situation 
        13. Change complies with target group needs 
        14. Competition with other (local) projects 
        15. Embed plan in existing activities 
    Development and selection of strategy for implementation and dissemination of the intervention 
        16. Plan is attractive for target group (pr) 
        17. Communication to target group 
    Development, testing, implementation 
        18. Has the implementation strategy been tested? 
        19. Developed on account of a problem instead of available financing 
        20. Monitoring of the implementation process 
        21. Adequate budget for implementation 
        22. Compensation/allowance performers 
        23. Change causes (temporary) inconvenience 
        24. Visible results 
        25. Involvement of the target group duringimplementation 
        26. Actual frequency of use compared to intended use 
        27. Time-bound implementation and planning 
        28. Time available for implementation 
    Evaluation 
        29. Process evaluation carried out? 
        30. Ability to customize change 
        31. Is the long term taken into account? 
Characteristics of the organization 
        32. Organization size (large, medium size, small) 
        33. Is there a clear direction? (leadership, budget) 
        34. Lack of staff 
        35. Collaboration with professionals in the neighbourhood 
        36. Previously existing local cooperation 
        37. Available expertise in the organization in relation to the intervention 
        38. Decision-making process/procedures in the organization (top-down/ bottom-up) 
Characteristics of the coordinator/project manager 
        39. Enthusiasm 
        40. Expertise related to the intervention 

Step 4: Assessment of the intervention’s RE-AIM by stakeholders

Policy makers (at municipal and neighbourhood levels), health behaviour change experts, public health service workers, neighbourhood opinion leaders and project leaders may all have a different view than the target group (the residents of underprivileged neighbourhoods) when assessing the success or failure of implementation. Both expert and target groups’ perspectives were considered to be of value in the assessment of the RE-AIM, and were assessed in panels (‘expert’ and target group panels).

Expert panels

Two expert group panels were organized, consisting of six individuals each (policy makers, academics, councillors, intervention developers and individuals from regional public health services and local healthcare services). Experts involved in the interventions subject to panel assessment were excluded from participation. During the panel meetings, handouts were provided with neighbourhood as well as city characteristics (i.e. number of inhabitants by race and age-group, level of education, employment status, income parameters and type of housing). The interventions were then systematically and objectively presented as cases, informing the experts about every intervention’s intended and actual targets, goals set and goals met, implementation facts and future prospects. This information was gathered using information that was made available through reports, e-mail correspondence and the interviews held during Step 3. The panel members assessed the degree to which they perceived the intervention to have been successful with regard to each of the five dimensions of RE-AIM, using a Likert scale (range 1–5). The panel members were blinded for the scoring of potentially important factors of implementation (as described in Step 3). This was done by refraining from providing any information as to whether the interventions addressed the theory-based factors.

Target group panels

The target group panels consisted of ‘end-users’ of the implemented environmental interventions under study: the residents of the three underprivileged neighbourhoods. Three users per intervention were asked in a face-to-face interview to assess the degree (1–5) to which they perceived the intervention to have been successful with regard to each of the five RE-AIM dimensions. For this purpose, the RE-AIM dimensions were converted to a comprehensible questionnaire that did not contain any technical terminology. Before its use, this questionnaire was pilot-tested for its comprehensiveness and clarity in a community centre in a different underprivileged neighbourhood than those under study, and was found to be suitable. The target group panels consisted mostly of women (82%), and 56% of participants were from an ethnic background other than Dutch, mainly Turkish (20%).

Statistical analyses

The factors potentially affecting implementation success or failure (from Step 3) were linked to the experts’ and the target group’s judgements on the RE-AIM dimensions. First, the mean scores on the separate RE-AIM dimensions were calculated and negatively worded items were reverse coded so that higher scores indicate a higher outcome on the different dimensions. Then, the Jonckheere–Terpstra exact test was performed to determine which factors potentially affecting success or failure were statistically associated with higher or lower scores on the different RE-AIM dimensions. The Jonckheere–Terpstra test is a non-parametric test for the existence of a trend between an ordinal grouping variable and another variable that is also distributed on at least an ordinal scale and was chosen because the data were not expected to be normally distributed. To reduce chance findings, the exact variant of the Jonckheere–Terpstra test was performed. All analyses were done separately for the expert and the target group, using SPSS 20.0 (SPSS Inc., Chicago, IL, USA).

The study protocol was approved by the Medical Ethics Committee of the VU University Medical Center in Amsterdam.

Results

Table 1 shows the interventions that were evaluated. One intervention (cycling lessons for immigrants by seniors) was excluded from the analysis because not enough data could be obtained from the intervention staff. Three other interventions were excluded because target groups did not assess the RE-AIM dimensions of these interventions (table 1; interventions 5, 13 and 18). The variance between interventions of six of the 40 factors potentially affecting implementation success or failure was too low to allow for the performance of the Jonckheere–Terpstra test (table 2; determinant 5, 35, 36, 37, 39 and 40).

The mean scores of all interventions for the five RE-AIM domains were deemed to be more positive by the target group than by the expert group (Supplementary Appendix 2). The mean scores of the target group on the Likert scales (range 1.0–5.0, with higher scores representing a worse outcome) were <2.5, whereas the mean scores of the expert group were >2.5 (Supplementary Appendix 2).

Factors associated with an increased RE-AIM

Of the 40 potential factors, six were statistically significantly associated with at least one of the RE-AIM dimensions scored by the stakeholder panels (table 3):

  • 1. 

    Target group involvement during process planning

  •    In interventions in which the target group was involved during the process planning, the actual effectiveness was higher and the implementation better, according to the experts.

  • 2. 

    Target group involvement during implementation

  •    When the target group was involved during the implementation phase, this was associated with a higher reach, effectiveness and adoption, according to the target and/or expert panels.

  • 3. 

    Inventory during development

  •    Interventions in which an inventory was carried out in the development phase (inventorying the target setting, context and problems and characteristics of the target group) had higher effectiveness, adoption and implementation, as judged by the experts.

  • 4. 

    Communication with target group before implementation

  •    Thorough communication with the target group during the development of an implementation strategy increased effectiveness, adoption and implementation.

  • 5. 

    Rivalry with similar interventions

  •    The experts also deemed maintenance to be higher when there was no competition with other (local) initiatives.

  • 6. 

    Intervention staffing

  •    The reach of the interventions was higher when there was sufficient intervention staff.

Table 3

Mean scores on RE-AIM outcomes of interventions (n = 17) when factors were addressed (yes) versus not (no), as assessed by expert (n = 12) and target group (n = 51) panels on a 5-point Likert scale

Factors Reach
 
Effectiveness
 
Adoption
 
Implementation
 
Maintenance
 
Target group Mean (SD) Expert group Mean (SD) Target group Mean (SD) Expert group Mean (SD) Target group Mean (SD) Expert group Mean (SD) Target group Mean (SD) Expert group Mean (SD) Target group Mean (SD) Expert group Mean (SD) 
Involvement of target group during process planning           
    Yes 2.1 (0.9) 2.8 (1.0) 2.1 (0.8) 2.7 (1.0) 2.2 (0.8) 2.3 (0.5) 1.8 (1.1) 3.0 (0.8) 1.8 (1.4) 2.6 (0.6) 
    No 2.4 (1.6) 3.4 (1.0) 3.1 (1.7) 3.9 (0.9) 2.4 (1.4) 3.0 (0.8) 2.4 (1.5) 3.7 (0.5) 3.0 (1.4) 3.4 (0.7) 
    Δ 0.3 0.6 1.1 1.2* 0.2 0.7 0.6 0.7* 1.2 0.8 
Inventory carried out during development           
    Yes 1.9 (0.8) 2.8 (0.8) 2.2 (1.0) 2.7 (0.8) 2.1 (0.6) 2.2 (0.4) 1.8 (0.8) 3.1 (0.7) 2.0 (1.2) 2.8 (0.7) 
    No 2.8 (1.5) 3.8 (1.1) 3.5 (1.7) 3.4 (0.6) 2.9 (1.5) 3.2 (0.6) 2.5 (1.8) 3.9 (0.4) 2.8 (1.7) 3.2 (0.6) 
    Δ 0.8 1.0 1.3* 1.6** 0.8 1.0** 0.6 0.8** 0.7 0.3 
Competition with other (local) projects           
    Yes 2.0 (0.6) 3.0 (1.1) 2.6 (1.5) 2.8 (1.1) 2.0 (0.2) 2.4 (0.8) 1.7 (0.3) 3.1 (0.8) 1.8 (0.9) 2.5 (0.5) 
    No 1.2 (0.0) 3.7 (0.0) 1.7 (0.0) 3.9 (0.0) 1.5 (0.0) 2.7 (0.0) 1.6 (0.0) 4.0 (0.0) 3.7 (0.0) 4.1 (0.0) 
    Δ −0.9 0.7 −1.0 1.0 −0.5 0.2 −0.1 0.9 1.9 1.6** 
Communication with target group           
    Yes 1.9 (0.8) 3.0 (0.9) 2.2 (1.0) 2.8 (0.9) 2.2 (0.6) 2.2 (0.5) 1.8 (0.8) 3.1 (0.7) 2.0 (1.2) 2.8 (0.7) 
    No 2.5 (0.0) 4.8 (0.0) 5.0 (0.0) 4.8 (0.0) 2.0 (0.0) 4.2 (0.0) 2.0 (0.0) 4.2 (0.0) 3.0 (0.0) 3.0 (0.0) 
    Δ 0.6 1.8 2.8 2.0* −0.2 1.9** 0.2 1.1* 1.0 0.2 
Involvement of target group during implementation           
    Yes 1.8 (0.8) 2.9 (0.9) 2.0 (0.7) 2.8 (0.9) 2.1 (0.8) 2.3 (0.4) 1.9 (0.9) 3.1 (0.7) 2.1 (1.3) 2.9 (0.8) 
    No 3.4 (1.4) 3.6 (0.9) 4.9 (0.2) 3.9 (1.2) 3.0 (1.7) 3.2 (1.1) 3.0 (1.7) 3.6 (0.6) 3.6 (1.2) 3.1 (0.8) 
    Δ 1.6* 0.7 2.9* 1.1* 0.9 0.9* 1.1 0.5 1.5 0.3 
Lack of staff           
    Yes 2.4 (1.1) 3.2 (1.0) 2.8 (1.4) 3.4 (1.1) 2.4 (1.0) 2.6 (0.7) 2.1 (1.2) 3.4 (0.7) 2.3 (1.4) 2.9 (0.6) 
    No  2.9 (0.0)  3.2 (0.0)  3.9 (0.0)  4.3 (0.0)  4.3 (0.0) 
    Δ NA −0.3** NA −0.2 NA 1.3 NA 0.9 NA 1.4 
Factors Reach
 
Effectiveness
 
Adoption
 
Implementation
 
Maintenance
 
Target group Mean (SD) Expert group Mean (SD) Target group Mean (SD) Expert group Mean (SD) Target group Mean (SD) Expert group Mean (SD) Target group Mean (SD) Expert group Mean (SD) Target group Mean (SD) Expert group Mean (SD) 
Involvement of target group during process planning           
    Yes 2.1 (0.9) 2.8 (1.0) 2.1 (0.8) 2.7 (1.0) 2.2 (0.8) 2.3 (0.5) 1.8 (1.1) 3.0 (0.8) 1.8 (1.4) 2.6 (0.6) 
    No 2.4 (1.6) 3.4 (1.0) 3.1 (1.7) 3.9 (0.9) 2.4 (1.4) 3.0 (0.8) 2.4 (1.5) 3.7 (0.5) 3.0 (1.4) 3.4 (0.7) 
    Δ 0.3 0.6 1.1 1.2* 0.2 0.7 0.6 0.7* 1.2 0.8 
Inventory carried out during development           
    Yes 1.9 (0.8) 2.8 (0.8) 2.2 (1.0) 2.7 (0.8) 2.1 (0.6) 2.2 (0.4) 1.8 (0.8) 3.1 (0.7) 2.0 (1.2) 2.8 (0.7) 
    No 2.8 (1.5) 3.8 (1.1) 3.5 (1.7) 3.4 (0.6) 2.9 (1.5) 3.2 (0.6) 2.5 (1.8) 3.9 (0.4) 2.8 (1.7) 3.2 (0.6) 
    Δ 0.8 1.0 1.3* 1.6** 0.8 1.0** 0.6 0.8** 0.7 0.3 
Competition with other (local) projects           
    Yes 2.0 (0.6) 3.0 (1.1) 2.6 (1.5) 2.8 (1.1) 2.0 (0.2) 2.4 (0.8) 1.7 (0.3) 3.1 (0.8) 1.8 (0.9) 2.5 (0.5) 
    No 1.2 (0.0) 3.7 (0.0) 1.7 (0.0) 3.9 (0.0) 1.5 (0.0) 2.7 (0.0) 1.6 (0.0) 4.0 (0.0) 3.7 (0.0) 4.1 (0.0) 
    Δ −0.9 0.7 −1.0 1.0 −0.5 0.2 −0.1 0.9 1.9 1.6** 
Communication with target group           
    Yes 1.9 (0.8) 3.0 (0.9) 2.2 (1.0) 2.8 (0.9) 2.2 (0.6) 2.2 (0.5) 1.8 (0.8) 3.1 (0.7) 2.0 (1.2) 2.8 (0.7) 
    No 2.5 (0.0) 4.8 (0.0) 5.0 (0.0) 4.8 (0.0) 2.0 (0.0) 4.2 (0.0) 2.0 (0.0) 4.2 (0.0) 3.0 (0.0) 3.0 (0.0) 
    Δ 0.6 1.8 2.8 2.0* −0.2 1.9** 0.2 1.1* 1.0 0.2 
Involvement of target group during implementation           
    Yes 1.8 (0.8) 2.9 (0.9) 2.0 (0.7) 2.8 (0.9) 2.1 (0.8) 2.3 (0.4) 1.9 (0.9) 3.1 (0.7) 2.1 (1.3) 2.9 (0.8) 
    No 3.4 (1.4) 3.6 (0.9) 4.9 (0.2) 3.9 (1.2) 3.0 (1.7) 3.2 (1.1) 3.0 (1.7) 3.6 (0.6) 3.6 (1.2) 3.1 (0.8) 
    Δ 1.6* 0.7 2.9* 1.1* 0.9 0.9* 1.1 0.5 1.5 0.3 
Lack of staff           
    Yes 2.4 (1.1) 3.2 (1.0) 2.8 (1.4) 3.4 (1.1) 2.4 (1.0) 2.6 (0.7) 2.1 (1.2) 3.4 (0.7) 2.3 (1.4) 2.9 (0.6) 
    No  2.9 (0.0)  3.2 (0.0)  3.9 (0.0)  4.3 (0.0)  4.3 (0.0) 
    Δ NA −0.3** NA −0.2 NA 1.3 NA 0.9 NA 1.4 

Notes: Values may range from 1.0 to 5.0, with higher scores representing worse performance.

Each of the 17 interventions was evaluated by six experts (two expert panel sessions were organized) and three target group individuals.

*P ≤ 0.05 (Jonckheere-Terpstra test).

**P ≤ 0.01 (Jonckheere–Terpstra test).

Discussion

In the current study, we investigated factors associated with the implementation of environmental interventions with a focus on promoting healthy behaviour in underprivileged neighbourhoods in The Netherlands. To our knowledge, this type of evaluation method (i.e. quantitative analyses of factors associated with five different outcomes in a range of interventions, as judged by both experts and target group) has not been used before. Six factors were identified that were significantly associated with better implementation, as indicated by RE-AIM dimensions. The results show that one or more dimensions were higher if (i) the target group was involved during planning and (ii) (in particular) during the implementation phase; (iii) an inventory was carried out during development; (iv) there was no competition with other (local) initiatives; (v) there was thorough communication with the target group (the potential intervention users) when developing an implementation strategy and (vi) there was sufficient intervention staffing.

This study contributes to the currently scarce evidence base for the effective implementation of public health measures focusing on physical and social environmental features to promote health behaviours. Previous studies that have dealt with comparable types of (complex) approaches, such as the Health Action Zones in the UK, support the expectation that a thorough evaluation is helpful in assessing and understanding these types of environmental change initiatives, even if the initiatives themselves do not appear to be completely successful.19 Acquiring more insight into how measures can be successfully implemented may improve future environmental interventions. Our findings that target group involvement may be of particular importance in various stages of the implementation process match with specific recommendations recently published in a public health guidance document by the National Institute for Health and Clinical Excellence.20

The relatively recent attention being paid to environmental-level determinants in addition to individual-level determinants of intervention success may be particularly valuable in efforts to reduce socioeconomic health inequalities.21 The strong link between health inequalities and the social and physical environment suggests that there is ample justification for intervening in underprivileged neighbourhood environments.22

Although environmental interventions are often implemented, their impact with regard to efficacy, effectiveness or other RE-AIM domains is seldom evaluated. In addition, investigation of factors that may affect this impact is usually ignored. This may also be due to the fact that, presumably, implementation research is not prioritized by stakeholders due to its complexity. Nonetheless, gaining knowledge about reach, adoption, implementation and maintenance of non-effective interventions remains to be of value, because it will help us to build an evidence base on what factors promote or hinder, or are at least associated with, such implementation and dissemination issues. The methodology used in the current study (linking potential theory-based factors with RE-AIM of interventions as assessed by a variety of stakeholder panels) proved suitable for identifying factors that are consistently associated with success. A further strength of this study is that the expert panel assessments of the interventions’ RE-AIM were ‘blinded’. Assessments were made based on information about the RE-AIM only, i.e. without provision of information as to whether potential important factors were addressed.

This study also had a number of limitations. First, panel judgements differed considerably across panels. In general, the expert panels were relatively negative about the RE-AIM of most interventions. In particular, they deemed the effectiveness to be very poor for interventions in which project leaders did not involve the target group. In line with the expert panels’ judgements, similar environmental interventions have been criticized in earlier studies for being short-term, unfocused and overly ambitious.22–24 The majority of the target group panels, on the other hand, were relatively positive about the RE-AIM dimensions of the same interventions, especially when they were involved in the implementation process. This positivity might also be a result of the fact that these individuals deliberately chose to participate in the interventions. Second, statistical significance was used to identify associations. Although the six identified factors demonstrated to have a consistent association with the scores of the target and/or expert panels, it is essential to realize that the statistical significance of associations does not necessarily imply that these factors are more important than others, or that they have a stronger association. More details and depth with regard to the importance of associations may be found by using a qualitative approach. Third, concerns may be raised about the possibility of chance finding, as a large number of variables were tested in our analyses (i.e. 40 potentially important factors were linked with five RE-AIM domains scored by expert as well as target group panels). To reduce chance findings, the exact variant of the Jonckheere–Terpstra test was performed. The results indicated that most ‘key factors’ identified in our study had a significant association with more than one RE-AIM domain (Supplementary Appendix 3), which strengthens the belief that these factors may have had an actual association. Fourth, questions could be raised about the likely generalizability of findings to other interventions in other neighbourhoods. However, due to the diversity of settings within the three study areas and the amount of relative similar settings in The Netherlands, the practical recommendations for implementation are likely to be of value in (deprived) neighbourhoods outside those under study. Fifth, the assessment of potentially important factors was done by two researchers who could not be fully blinded with regard to outcomes of the intervention. The researchers had to gather relevant information about each intervention and held interviews with the intervention developers and providers. This way, the researchers may have obtained an idea with regard to the intervention’s effectiveness while assessing the potentially important factors. Finally, because the characteristics of the coordinators/team managers had not enough variance between interventions, we were not able to test their associations with intervention outcomes.

In conclusion, this study revealed specific factors associated with an increased RE-AIM. Although the relative importance of these factors requires further investigation, it is recommended that they should be addressed in planning, development and implementation of future environmental interventions to improve health behaviours. The results stress the importance for intervention implementers to involve the target group in various stages of the implementation process, make an inventory of the situation before implementation, refrain from competition with similar local projects and, finally, ensure enough staffing.

Funding

This work was supported by the Dutch Diabetes Research Foundation [grant number 2010.104.1354].

Conflicts of interest: None declared.

Supplementary data

Supplementary data are available at EURPUB online.

Acknowledgements

The authors thank Evelyn Mus-Mulder for her support in collecting the data.

Key points

  • It is well-established that there is a link between social and physical environment on one hand and health-related behaviour on the other.

  • To promote healthy behaviour by improving the social and physical environments in underprivileged neighbourhoods, many interventions are carried out.

  • However, enhancing or impeding factors associated with successful implementation of such interventions are rarely studied.

  • This study contributes to the evidence base for the effective implementation of social and physical environmental interventions to promote healthy behaviour in underprivileged neighbourhoods.

  • In particular, the results highlight the importance of involving the target group in various stages of the implementation process.

References

1
Giskes
K
Avendano
M
Brug
J
Kunst
AE
A systematic review of studies on socioeconomic inequalities in dietary intakes associated with weight gain and overweight/obesity conducted among European adults
Obes Rev
 , 
2010
, vol. 
11
 (pg. 
413
-
29
)
2
Kamphuis
CB
van Lenthe
FJ
Giskes
K
, et al.  . 
Socioeconomic differences in lack of recreational walking among older adults: the role of neighbourhood and individual factors
Int J Behav Nutr Phys Act
 , 
2009
, vol. 
6
 pg. 
1
 
3
Mackenbach
JP
Stirbu
I
Roskam
AJ
, et al.  . 
Socioeconomic inequalities in health in 22 European countries
N Engl J Med
 , 
2008
, vol. 
358
 (pg. 
2468
-
81
)
4
Mackenbach
JP
Howden-Chapman
P
New perspectives on socioeconomic inequalities in health
Perspect Biol Med
 , 
2003
, vol. 
46
 (pg. 
428
-
44
)
5
Hawkins
N
Jhund
P
McMurray
JJ
Capewell
S
Heart failure and socioeconomic status: accumulating evidence of inequality
Eur J Heart Fail
 , 
2012
, vol. 
14
 (pg. 
138
-
46
)
6
Ball
K
Timperio
A
Salmon
J
, et al.  . 
Personal, social and environmental determinants of educational inequalities in walking: a multilevel study
J Epidemiol Community Health
 , 
2007
, vol. 
61
 (pg. 
108
-
14
)
7
Bot
S
Lakerveld
J
Kingo
L
, et al.  . 
The association between social network characteristics and heatlhy lifestyle behaviour
Forthcoming
8
Lindstrom
M
Hanson
B
Ostergren
P
Socioeconomic differences in leisure-time physical activity: the role of social participation and social capital in shaping health related behaviour
Soc Sci Med
 , 
2001
, vol. 
52
 (pg. 
441
-
51
)
9
Lovasi
GS
Hutson
MA
Guerra
M
Neckerman
KM
Built environments and obesity in disadvantaged populations
Epidemiol Rev
 , 
2009
, vol. 
31
 (pg. 
7
-
20
)
10
Sallis
J
Floyd
M
Rodriguez
D
Saelens
B
Role of built environments in physical activity, obesity, and cardiovascular disease
Circulation
 , 
2012
, vol. 
125
 (pg. 
729
-
37
)
11
Lovasi
GS
Hutson
MA
Guerra
M
Neckerman
KM
Built environments and obesity in disadvantaged populations
Epidemiol Rev
 , 
2009
, vol. 
31
 (pg. 
7
-
20
)
12
Mackenbach
J
Can we reduce health inequalities? An analysis of the English strategy (1997-2010)
J Epidemiol Community Health
 , 
2011
, vol. 
65
 (pg. 
568
-
75
)
13
Judge
K
Politics and health: policy design and implementation are even more neglected than political values?
Eur J Public Health
 , 
2008
, vol. 
18
 (pg. 
355
-
6
)
14
Glasgow
RE
Vogt
TM
Boles
SM
Evaluating the public health impact of health promotion interventions: the RE-AIM framework
Am J Public Health
 , 
1999
, vol. 
89
 (pg. 
1322
-
7
)
15
Lakerveld
J
Verstrate
L
Bot
S
, et al.  . 
Social and physical environmental interventions in low socio-economic neighbourhoods to promote healthy behaviours: determinants of implementation and use
Int J Patient Cent Med
 , 
2012
, vol. 
2
 (pg. 
491
-
5
)
16
Patton
M
Qualitative research and evaluation methods
 , 
2002
Thousand Oaks, CA
Sage Publications, Inc.
17
Grol
R
Wensing
M
Implementatie: effectieve verbetering van de patiëntenzorg [Implementation: effective changes in patient care]
 , 
2006
2nd edn
Maarssen
Elsevier gezondheidszorg
18
Fleuren
M
Wiefferink
K
Paulussen
T
Determinants of innovation within health care organizations: literature review and Delphi study
Int J Qual Health Care
 , 
2004
, vol. 
16
 (pg. 
107
-
23
)
19
Judge
K
Bauld
L
Learning from policy failure? Health action zones in England
Eur J Public Health
 , 
2006
, vol. 
16
 (pg. 
341
-
3
)
20
 
The National Institute for Health and Clinical Excellence (NICE). Obesity: working with local communities, 42, 2012
21
Lakerveld
J
Brug
J
Bot
S
, et al.  . 
Sustainable prevention of obesity through integrated strategies: the SPOTLIGHT project's conceptual framework and design
BMC Public Health
 , 
2012
, vol. 
12
 pg. 
793
 
22
Thomson
H
A dose of realism for healthy urban policy: lessons from area-based initiatives in the UK
J Epidemiol Community Health
 , 
2008
, vol. 
62
 (pg. 
932
-
6
)
23
Ho
SY
Evaluating urban regeneration programmes in Britain
Evaluation
 , 
1999
, vol. 
5
 (pg. 
422
-
38
)
24
Lawless
P
Area-based urban interventions: rationale and outcomes: the new deal for communities programme in England
Urban Stud
 , 
2006
, vol. 
43
 (pg. 
1991
-
2011
)

Comments

0 Comments