Organizational readiness and implementation fidelity of an early childhood education and care-specific physical activity policy intervention: findings from the Play Active trial

Abstract Background Many children do not accumulate sufficient physical activity for good health and development at early childhood education and care (ECEC). This study examined the association between ECEC organizational readiness and implementation fidelity of an ECEC-specific physical activity policy intervention. Methods Play Active aimed to improve the ECEC educator’s physical activity practices. We investigated the implementation of Play Active using a Type 1 hybrid study (January 2021–March 2022). Associations between organizational readiness factors and service-level implementation fidelity were examined using linear regressions. Fidelity data were collected from project records, educator surveys and website analytics. Results ECEC services with higher levels of organizational commitment and capacity at pre-implementation reported higher fidelity scores compared to services with lower organizational commitment and capacity (all Ps < 0.05). Similarly, services who perceived intervention acceptability and appropriateness at pre-implementation to be high had higher fidelity scores (P < 0.05). Perceived feasibility and organizational efficacy of Play Active were associated with higher but nonsignificant fidelity scores. Conclusions Results indicate that organizational readiness factors may influence the implementation of ECEC-specific physical activity policy interventions. Therefore, strategies to improve organizational readiness should be developed and tested. These findings warrant confirmation in the ECEC and other settings and with other health behavior interventions.


Background
2][3][4][5][6] For physical activity and sedentary behaviors, these Guidelines [7][8][9][10] state that, each day, children aged 2-5 years should accumulate ≥180 minutes of total physical activity, including ≥60 minutes of moderate to vigorous physical activity for 3-5-year-olds, and ≤60 minutes of sedentary/screen time.][9][10] A lack of sufficient physical activity and high levels of sedentary time in young children can lead to an increased risk of high blood pressure, insulin resistance, musculoskeletal problems, obesity, bullying and being socially isolated. 7,9rly childhood education and care (ECEC) is an ideal setting for interventions aimed at increasing young children's physical activity, with between 50 and 90% of 0-5-year-old children attending out-of-home care globally. 11,12However, many young children do not accumulate sufficient physical activity for good health and development while at ECEC. 12 Although increased levels of physical activity have been reported in some ECEC-based interventions, these results are often marginal and fade quickly. 12,13The potential for positive longer-term impact on physical activity practices through better organizational support and implementation strategies has yet to be explored.
ECEC educators are pivotal in shaping young children's behaviors. 13However, competing educator work priorities can be a barrier when implementing changes in policy or practice.Few studies have focused on the key contextual, organizational and individual barriers to implementation of physical activity interventions in ECEC. 12,14However, a lack of training, negative educator mindset, lack of resources, parent and educator risk aversion, dissent about physical activity as a learning experience, 15 limited technological capacity and staff turnover 16 have been reported as implementation barriers to a new ECEC-specific physical activity policy. 15While addressing these barriers is an important step, there is no guarantee that ECEC-specific physical activity interventions will be translated into practice by educators if services are not adequately prepared and 'ready' to implement. 15urrently, organizational readiness is not commonly measured for any type of intervention in the ECEC setting. 17Organizational readiness is a multifaceted concept, evident across all levels of an organization and comprises notions of shared resolve, motivation and collective capacity to undertake practice change. 188][19][20][21][22] Preimplementation knowledge about the perceived acceptability, appropriateness and feasibility of an intervention by an organization is also recognized as useful for the early mitigation of implementation barriers. 23,24While often used as outcome measures of implementation quality, 23 we propose that 'organizational readiness' is a concept that should include the pre-implementation knowledge of intervention acceptability, appropriateness and feasibility.Acceptability defines how agreeable an intervention is perceived to be, whereas appropriateness denotes the perceived intervention relevance. 24Feasibility is the perceived extent to which the intervention can be implemented. 24In effect, perceived acceptability, appropriateness and feasibility are organizational characteristics that indicate the compatibility of an intervention with a particular organizational setting. 25][25] Organizational readiness influences intervention implementation success. 26,27In the ECEC setting, these factors that can help with ensuring educators are 'on the same page' as their service director/leader, 15,28 can help overcome imple-mentation barriers by increasing intervention knowledge, promote educator participation and support adoption of new practices by helping educators feel confident in their abilities to adapt to new policies and procedures. 29Given this, further knowledge of organizational readiness factors as tools for supporting the implementation of ECEC-specific physical activity interventions requires investigation.
Organizational readiness is critical to implementation fidelity. 30,31Implementation fidelity explains 'what works' in effectiveness trials, 32,33 informs study replication, 33 supports practical application 32,33 and identifies protocol variations. 33he measurement and reporting of implementation fidelity can provide information about whether implementation strategies are sufficient and if additional or different strategies are needed. 34A recent review found that the level of implementation of ECEC-specific policies and practices were low and implementation strategies were insufficient to change these outcomes. 12Measuring implementation fidelity, therefore, is necessary for understanding the success of ECEC-specific physical activity interventions.Further research is needed to understand the specific organizational readiness factors that influence the implementation fidelity of health interventions in this setting.The aim of this study, therefore, was to examine the associations between ECEC organizational readiness factors and the implementation fidelity of an ECEC-specific physical behavior policy intervention-Play Active.

Methods
A pragmatic cluster randomized trial evaluated the effectiveness of the Play Active physical activity policy intervention to improve ECEC educators physical activity-related practices. 35he primary outcome of Play Active was reported change in educator physical activity-related practices and is described in detail in the published protocol. 35This study measured the implementation fidelity of Play Active as a secondary outcome using a Type 1 Hybrid effectiveness-implementation design. 36This study design maintains the focus on intervention effectiveness outcomes while allowing exploration of the intervention's implementation. 36lay Active is an evidence-informed ECEC-specific physical activity policy template with nine physical activity and sedentary time recommendations for young children in care. 35The policy intervention is underpinned by six implementation strategies (implementation intervention): (i) personalization of the policy, (ii) policy review and approval, (iii) resource guide mapped to policy practices, (iv) Energetic Play Assessment Tool (EPAT) to monitor young children's physical activity levels, (v) online professional development and (vi) technical assistance for implementation support. 35fter tailoring the Play Active policy template to suit service needs, directors completed a pre-implementation survey.A 3-5-month period was provided for the implementation of Play Active, after which service directors completed a post-implementation (follow up) survey.Ethics approval was provided by the University of Western Australian Human Ethics Research Committee (RA/4/20/6120).

Participants and setting
Long day ECEC services caring for children aged 0-5 years with a minimum 20 enrolled children and located in the Perth, Australia metropolitan area were eligible to take part.Services were invited to complete an expression of interest (EOI) to participate.Information and consent packs were provided to each service.Consenting services were randomly allocated to either the intervention group (n = 40) or the wait-listed control group (n = 40).Only intervention services participated in the current study.

Data collection and measures
Service directors completed a survey in the period after tailoring their physical activity policy and prior to commencing implementation (pre-implementation period: January to June 2021).The post-intervention survey was collected from September 2021 to March 2022 (post-implementation period).In this study, data were either collected at the servicelevel or summarized to the service-level for consistency.

Demographic measures
Service-level demographics were collected at pre-implementation.Directors' level of education was coded into two categories (lower: secondary school/certificate/diploma versus higher: university degree).Service-level data were sourced from the Australian Children's Education and Care Quality Authority (ACECQA) website 37 and included service ratings for the 'Quality Rating 2: Children's health and safety' element: 'Each child's health and physical activity is supported and promoted' (ratings: not assessed, working toward, meeting and exceeding) and number of approved places to determine service size (small/medium ≤57 children, large = 58-74 children and very large >75 children).Service suburb postcode was used as a proxy for socio-economic status (SES) (low, middle and high) and derived from the Australian Bureau of Statistics' Socio-Economic Indexes for Areas (SEIFA). 38

Organizational readiness measures
The 12-item Organizational Readiness for Implementing Change (ORIC) 39 measured ECEC service change com-mitment and change efficacy at pre-implementation (see Supplementary Table 1: Organizational Readiness Measures).0][41] and suitable for use at the organizational level. 39rganizational capacity was measured at pre-implementation by the 5-item organizational capacity scale from the Program Sustainability Assessment Tool (PSAT) (see Supplementary Table 1). 42The PSAT tool is reliable, with the organizational capacity scale reporting α = 0.87 for internal consistency. 42he Acceptability of Intervention Measure (AIM), Intervention Appropriateness Measure (IAM) and Feasibility of Intervention Measure (FIM) scales, developed by Weiner and colleagues, 24 measured directors' perceptions of the acceptability, appropriateness and feasibility of Play Active at preimplementation (Supplementary Table 1).The AIM, IAM and FIM 4-item scales are structurally valid (AIM α = 0.85, IAM α = 0.91, FIM α = 0.89), reliable (test-retest: AIM α = 0.83, IAM α = 0.87, FIM α = 0.88) and designed to be customisable. 24ue to > 95% of responses falling into the two highest of the five response for each of organizational commitment, efficacy, capacity and intervention acceptability, appropriateness and feasibility scales, responses were recoded into two categories as shown in Table 1.

Fidelity measure
Informed by the work of Proctor et al . 23a fidelity measure was constructed to measure overall implementation fidelity, adherence, dose, quality of delivery and participant responsiveness scores of Play Active at post-implementation (Fig. 1).These were based on fidelity indictors set out in the Play Active protocol (e.g.personalize policy: services select at least five from 25 practices within the physical activity policy template to focus on initially). 35Sources for fidelity indicator data included web analytics, selected educator evaluation survey questions and project administration records.The components of the fidelity measure items were converted to z-scores to produce individual scores for adherence, dose, quality of delivery and participant responsiveness and then summed to produce an overall fidelity score (Supplementary Table S2).

Data analysis
Using SPSS v.28, 43 descriptive statistics were calculated for all variables including frequencies and percentages.Independent t-tests were used to analyze unadjusted associations that compared the overall fidelity score between those with, e.g.some or neutral commitment against those who were committed, for each independent variable (organizational com-

Results
Most ECEC intervention services (73%) were large to very large with 58 or more approved places for children (Table 2).Around half of services (48%) were in high SES areas and just over half (55%) were rated as 'meeting the national quality standard' for 'Quality Rating 2: Children's health and safety', which includes promoting children's physical activity. 44The majority of ECEC service directors (80%) did not hold tertiary education qualifications.Most directors reported their ECEC service had commitment (61.1%) and efficacy (72.2%) in place to implement Play Active whereas about one-third (30.6%) reported having the organizational capacity to change to a 'very great' extent.While in pre-implementation, services were almost equally split on the level of acceptability of Play Active (somewhat acceptable/neutral 41.7% versus acceptable 58.3%).Most reported that the intervention was appropriate and/or feasible (both 74.3%).
The overall fidelity mean z-score for ECEC services was 1.7 ± 5.0 (possible range: −7.97 to 12.33) at the 3-5-month follow-up.The fidelity components of adherence, dose, quality of delivery and participant responsiveness were examined by service demographic variables (Supplementary Fig. 1a and b).More ECEC services met the dose criteria in the 'exceeding' quality rating for Children's Health and Safety (67%) compared with those who had a 'meeting' quality rating (25%) or were 'working toward'/'not assessed' (38%) (P = 0.027).More small/medium (91%) or large ECEC services (92%) met the participant responsiveness criteria than very large services (63%) (P < 0.048).No other significant socio-demographic differences were found.
Unadjusted associations between the six organizational readiness baseline variables and the post-implementation fidelity score reported significant results for all except feasibility (Supplementary Table 3).

Adjusted associations between organizational readiness and Play Active implementation fidelity
After adjusting for service-level characteristics, ECEC services who were fully 'committed' to implementing Play Active reported significantly higher fidelity than services with 'some or neutral commitment' (MD 4.96; 95% CI 1.62-8.29;P = 0.005) (Table 3).Similarly, ECEC services who had organizational capacity to a 'very great extent' reported significantly higher fidelity scores than services with less than to a 'very great extent' (MD 4.11; 95% CI 0.78, 7.43; P = 0.017).No significant results were found   between organizational efficacy and fidelity; however, higher fidelity scores were seen in services reporting Play Active as 'efficacious' over those reporting 'some or neutral efficacy'.ECEC services who reported Play Active was acceptable or appropriate at pre-implementation had significantly higher fidelity scores compared to services with lower levels of perceived acceptability or appropriateness (MD 4.02, 95% CI 0.63-7.40,P = 0.022, and MD 5.13, 95% CI: 1.32-8.93,P = 0.010, respectively) (Table 3).The perceived feasibility of Play Active was associated with higher but non-significant fidelity scores.

Main findings of the study
This study examined the association between ECEC service organizational readiness factors and the implementation fidelity of Play Active.Higher levels of Play Active implementation fidelity were associated with higher levels of organizational commitment and organizational capacity for change.Our results also showed that ECEC services with higher implementation fidelity of Play Active had higher levels of pre-implementation intervention acceptability and appropriateness.Overall, implementation fidelity to Play Active was positively associated with factors of organizational readiness.

What is already known on this topic
45,46 More recently, Metz and colleagues 47 used an ECEC-specific lens with implementation science concepts to suggest that higher levels of organizational readiness may point to the development of an 'hospitable environment' 47 for successful implementation of, and fidelity to, evidence-based interventions in the ECEC setting.In addition, previous work has examined the use of strategies to increase acceptability and appropriateness of childhood obesity interventions. 23,48Improving intervention 'fit' at the service-level can be facilitated by strategies such as pre-intervention training and planning to engage and incorporate expertise of potential end-users (i.e.ECEC staff).This promotes ownership of practice change and can ensure implementation aligns with the setting's needs, priorities and available resources. 15,28,29,45The use of such strategies is important given an increasing focus on 'designing for dissemination and implementation' 24 and requirements for the substantial time and costs associated with intervention development to represent value for money. 45

What this study adds
The purposeful creation of an hospitable environment through organizational readiness strategies (e.g.two-way communication, quality improvement processes and time for professional development) in ECEC services may be necessary to support end-user engagement with implementation, 49 promote intervention fidelity 46 and to support ownership of the resultant practice changes. 45Play Active's use of such strategies (e.g.educator engagement in Play Active development) may have facilitated the high levels of commitment, capacity, acceptability and appropriateness reported.Practical strategies that may increase organizational readiness in ECEC include the provision of sufficient educators to cover for those undertaking professional development, more/new play equipment, methods in place to track changes in children's physical activity and early engagement of parents/carers and other stakeholders, 47 talking with educators regularly (informally and formally) about the new policy they will implement and getting educator input as to their support needs for implementation success. 15verall, our findings provide support for the relationships between organizational readiness factors and implementation fidelity.However, further research is required to clarify these relationships and develop strategies to improve organizational readiness prior to implementation.
These results may have been influenced by the timing of the Play Active intervention.Play Active was implemented during the Covid-19 pandemic, which involved various restrictions (e.g.fewer children in care due to mandatory lockdown).Higher levels of staff turnover, increased sick leave, isolation restrictions, longer work hours, stress and burnout among ECEC staff resulted in less time for planning or implementing 50 a new physical activity policy.Despite this, our results indicate organizational readiness is key to implementation fidelity for ECEC-specific physical activity policy, however, further research is needed to confirm these findings post-Covid-19 restrictions.

Limitations of the study
The Play Active trial was subject to limitations that impacted the current study, including: a relatively short implementation period, self-selection of the ECEC sample through the EOI process and ECEC workforce challenges exacerbated by Covid-19 restrictions.Participating ECEC services were metropolitan only, impacting the generalizability of our results for implementing Play Active in non-metropolitan areas.Minimal variation across some survey item responses resulted in response scales being dichotomized for data analysis.In addition, the measures used were either from other settings or were developed for the current study and therefore were not validated in ECEC settings.ECEC service visits were forbidden during the implementation period (Covid-19) and relied on service director self-reports rather than objective observations of the implementation of Play Active.Finally, as only long daycare services participated in the main study, these results may not be applicable to other ECEC types (e.g.family day care and out of school care).

Conclusions
This study explored the relationships between organizational readiness and implementation fidelity of the ECEC-based Play Active physical activity policy intervention.Our results demonstrated that increases in Play Active implementation fidelity were significantly associated with higher levels of organizational readiness factors.These results are important for informing the future implementation and sustainability of physical activity interventions in the ECEC setting as many rarely make significant or lasting increases in children's activity levels.To support successful implementation and the subsequent positive impact on child physical activity levels and health, further research should be undertaken to determine how to develop and incorporate organizational readiness strategies into future ECEC-specific physical activity policy interventions.

( 2 )Fig. 1
Fig. 1 Development of fidelity measure.Program differentiation not measured.PD, professional development; Pre, pre-implementation data collection; Post, post-implementation data collection.

Table 1
Recoding of organizational readiness response variables for analysis

Table 2
ECEC service sample characteristics a Missing = 4.b Mean ± standard deviation.

Table 3
Multivariable linear regression of Play Active fidelity score and baseline organizational readiness