Introduction

Organization-based health programs require the collaboration and support of the people within organizations. This is especially true for structural interventions, characterized by changes at the organizational level as a key component of the program. Structural interventions in health promotion seek to change the social and built environments of organizations (Cohen et al. 2000). Furthermore, this type of intervention is optimally carried out in partnership between researchers and community organizations (Cohen et al. 2000), and is implemented by the organization staff. Therefore, structural interventions are dependent upon intensive collaboration with leadership and stakeholders. For this reason, organizational or leadership support can impact the degree to which interventions are implemented. Changing the built or social environments of organizations or their policies and practices requires administrative support beyond permission to provide individual- or small group-level educational sessions.

Organizations typically differ on their level of implementation (Saunders et al. 2013), with variation based on organizational characteristics, provider and program characteristics, implementation processes, and community-level factors (Dominick et al. 2014; Durlak and DuPre 2008). Imperfect implementation is ubiquitous and influenced by many factors, including organizational support for implementation (Durlak and DuPre 2008). Similarly, high perceived organizational support more broadly often results in a positive reciprocal relationship of commitment, and thus moving the organization toward its mission (Rhoades and Eisenberger 2002).

Organizational characteristics include organizational climate and support from supervisors and administrators, defined as the value placed on employees and their work activities (Durlak and DuPre 2008; Rhoades and Eisenberger 2002; Scaccia et al. 2015). Organizational climate, defined by perceived support for the program, staff buy-in, and dedicated resources, is an essential element of effective programs (Durlak and DuPre 2008; Scaccia et al. 2015). High perceived organizational support often results in a positive reciprocal relationship of commitment and moving the company toward its mission (Rhoades and Eisenberger 2002). In structural interventions, the quality of the relationship between the external facilitators and researchers with the organization also is important.

Organizational support is important for implementation of structural interventions in schools as well (Kelder et al. 2003; Thapa et al. 2013) and other settings (Dominick et al. 2014; Saunders et al. 2013). Teachers who believe they are supported are more likely to be dedicated to being a good teacher, report less burnout, and have higher retention rates (Thapa et al. 2013). Evidence from the education and school climate literature suggests that principal, or leadership, support functions similarly to organizational support. Support from school administrators or employees’ supervisors is an integral component of successful health promotion programs (Birken et al. 2013). Successful implementation of health promotion programs in these and other settings likely is dependent in part upon an organization’s capacity to carry out the planned intervention components (Durlak and DuPre 2008; Scaccia et al. 2015). This may be especially true for structural interventions because they are dependent upon the setting and context of program implementation. As more interventions are using process evaluation to measure implementation fidelity, tracking determinants of high implementation is important, including the role of organizational or administrative support, particularly in non-school settings.

Health promotion programs that intervene at the organizational or community level are becoming more common. Organizational support is a frequently cited determinant of higher levels of implementation fidelity, which can impact behavioral outcomes. In this study, organizational support is operationally defined as awareness of the intervention activities, reaction to the intervention activities, and involvement of other staff and administrators in the intervention activities. This article describes the impact of organizational support on program fidelity, and subsequently on changes in eating fruits and vegetables (FV; Durlak and DuPre 2008) among children living in residential children’s homes (RCH) using a mediation model. The Environmental Interventions in Residential Children’s Homes (ENRICH) project was a 5-year project with goals to increase physical activity and healthy eating among children living in RCH through creating health-promoting home environments (Berta et al. 2015). Level of implementation across participating homes (Saunders et al. 2013) and the impact of implementation fidelity on physical activity outcomes have been reported (Dominick et al. 2014; Lau et al. 2016). However, implementation of ENRICH in the different RCHs was different for the nutrition components compared with the physical activity elements (Saunders et al. 2013).

Therefore, the purposes of this study were to (1) report the change in FV intake among children living in the RCH in the intervention and control (i.e., delayed intervention) groups, (2) evaluate the association between change in organizational support over the course of the 2-year intervention with nutrition implementation fidelity at post-test, and (3) assess whether change in organizational support moderated implementation fidelity or was an antecedent in a mediation model explaining the impact of implementation fidelity on nutrition outcomes (child, organization). Children in the USA do not meet national guidelines for FV intake of 3–5 cups on average (Moore et al. 2017), highlighting the need for health promoting interventions in this age group. Examining the implementation of nutrition components in ENRICH contributes to the evidence on efficacy and effectiveness of nutrition-based programs for youth.

Methods

Study Setting and Participants

ENRICH was implemented in residential children’s homes (RCHs) in North and South Carolina. These group homes are non-familial and provide shelter and care to children in their area. Homes also may offer additional programming such as social services and mental healthcare. The size of homes varies widely, as does their participation in the National School Breakfast and School Lunch programs (64% of the sample participated). As reported in Dominick et al. (2014), inclusion criteria for RCH participation in ENRICH was not being a provider of emergency services only, and primarily serving children with no restrictions on physical activity participation or on nutritional intake.

ENRICH utilized a randomized cross-over design and included longitudinal organizational assessments and repeated cross-sectional assessments at the child-level, similar to school-based interventions (Foster et al. 2008; Stevens et al. 2005). Participating RCHs were part of a group of 63 licensed RCHs in North and South Carolina affiliated with The Duke Endowment. Of the 54 RCHs that agreed to participate in an organizational intervention, 29 were eligible to participate based on stability of youth population and management needs of the children. RCHs (n = 29) were randomized by the investigators at the RCH-level into early-intervention (n = 17) or control (n = 12) groups after being matched on organizational characteristics such as state (North Carolina or South Carolina), participation in the National Breakfast and Lunch Program, and number of locations (Dominick et al. 2014).

Data from children ages 11 to 18 years living in RCHs were collected at three cross-sectional waves: 2004, 2006, and 2008 (n = 614; 53.3% in intervention group), where 2006–2008 was the control period for the control group. Longitudinal follow-up at the individual level was not possible due to the transient nature of the resident population for whom average stay duration was less than 1 year. Children from the three cross-sectional waves of data collection (n = 614) were included in the analyses. The sample was 52.4% female, with a mean age of 14.54 years (± 1.89) and a mean body mass index (BMI) of 24.42 (± 6.38). More than half of the children were White (51.3%), 27.0% were Black/African American, and 21.7% reported other or multiple races/ethnicities.

ENRICH Intervention

ENRICH was a structural intervention (Cohen et al. 2000) that aimed to improve the physical activity participation and FV consumption of children living in the participating RCHs. Participating RCHs created or built on an existing Wellness Team to plan and implement changes to the built and social environments using a facilitation approach (Berta et al. 2015). ENRICH staff supported the RCHs and Wellness Teams by providing ongoing in-service training, workshops, and technical assistance. In both the intervention and delayed intervention groups, Wellness Teams were tasked with developing objectives to increase availability of FV, provide encouragement and role modeling for children to eat FV, post informational materials around the RCH about eating FV, and collaborate with outside organizations to promote and provide FV after receiving training and tools to facilitate planning for RCH environment change (Evans et al. 2009; Saunders et al. 2013). Although they were guided by common project goals, the exact nature of the environmental and policy changes was determined by the Wellness Team in each home. Example environmental changes included FV were made available for snacks, staff encouraged eating FV, policies were enacted to provide FV at each meal, and allowing staff time to plan for child nutrition (Evans et al. 2009). Details regarding the environmental changes for physical activity and related outcomes are reported elsewhere (Dominick et al. 2014; Lau et al. 2016). The implementation monitoring process was guided by a systematic process based on process evaluation approaches in public health (Baranowski and Stables 2000; Saunders et al. 2005; Steckler and Linnan 2002). The process evaluation framework and objectives for this project are described elsewhere (Saunders et al. 2013).

Measures—Child Level

Cross-sectional assessments of anthropometric and demographic variables and FV intake were conducted in 2004, 2006, and 2008. Height and weight were measured by ENRICH staff, and BMI was calculated. Children self-reported their age, sex, and race/ethnicity. Duration of time spent in the home for each child was reported by staff at each RCH. There were no differences in race, age, or BMI at baseline. There was a greater proportion of children who were male in the intervention condition than the control condition at baseline (54.4% vs 36.1%, P = 0.013).

FV intake was measured via two telephone-administered 24-h dietary recalls using the multi-pass approach (Baxter et al. 2003; Kristal and Satia 2001). Telephone calls were performed by a registered dietitian on random days (one weekday and one weekend day). Data collectors provided participants with a validated food portion visual during anthropometric and demographic data collection. No differences were detected between weekday and weekend day FV consumption (Evans et al. 2009). Average daily servings of FV were calculated using the Nutrient Data System for Research (version 2005) software.

Measures—Organizational Level

In addition to collecting data from children living in the RCHs, fidelity of intervention implementation and organizational support data were collected from staff at the RCHs, including the Wellness Team Coordinator. All procedures and data collection methods were approved by the Institutional Review Board at the researchers’ institution and the individual RCHs.

Organizational Environment Characteristics

Study staff assessed intervention implementation as changes in the built and social environment that were supportive of serving and consuming FV in the RCHs. Multiple data sources (i.e., Wellness Team contact report, ENRICH staff rating, and environmental observation) allowed for triangulation of implementation data and have been described in detail previously (Evans et al. 2009; Saunders et al. 2013). Briefly, environmental characteristics for nutrition implementation fidelity included the extent to which (1) the RCH was meeting the nutrition objectives set by the wellness team at the beginning of the intervention; (2) the RCH made FV available at meal and snack times; (3) the RCH was able to identify and use external nutrition resources; (4) consumption of FV was promoted through media (e.g., posters) at the RCH; (5) adults in the RCH modeled or encouraged eating FV; and (6) the recommended number of servings were provided to children living in the RCH (measured as ≥ 3 servings available at dinner; Saunders et al. 2013). All measures were assessed annually except for whether the nutrition objectives were met, which was evaluated at the end of the 2-year intervention period (Saunders et al. 2013). A composite implementation fidelity score used the continuous values from each of the six elements to calculate the mean score for implementation at the home level. This summary score of implementation fidelity, where higher scores represent greater implementation, was selected to provide a numeric dependent variable for the analyses, and is described in detail in Saunders et al. (2013). The social environment for FV was higher in the intervention RCHs at baseline relative to controls, but no other organizational differences were detected, including provision of servings of FV.

Staff and Leadership Awareness, Reaction, and Involvement

Support for the intervention was measured via a phone interview with the Wellness Team Coordinator. The survey was conducted at the end of each program year and included items on awareness of Wellness Team members, supervisors, and the CEO at the RCH about the Wellness Team activities (3 items; e.g., How aware of the Wellness Team activities was your direct supervisor?) with response options of “very aware” to “completely unaware,”, reaction of Wellness Team members, direct supervisors, and the CEO to the ENRICH intervention (3 items; e.g., How did your direct supervisor react to the Wellness Team activities?) with response options of “very supportive and positive” to “very unsupportive and negative,” and involvement of Wellness Team members, supervisors, and the CEO in Wellness Team or other intervention activities (3 items; e.g., How involved was your direct supervisor in Wellness Team activities?) with response options of “very involved” to “completely uninvolved.” The Cronbach’s alpha values for these constructs were 0.81 for awareness, 0.63 for react, and 0.78 for involvement. Sum scores were calculated to represent total reaction, involvement, and awareness by RCH.

Analysis Plan

Descriptive statistics were calculated for baseline and follow-up child-level variables of age, sex, race/ethnicity, and average number of FV servings consumed. A two-way (condition by time) repeated measures ANOVA was conducted to test whether FV intake increased in response to the intervention. Children were nested within RCHs. The model controlled for child race/ethnicity, sex, age, BMI, and time the child had been at the RCH.

A multilevel regression model (design effect > 2) was used to determine whether the association between implementation fidelity and FV intake was moderated by organizational support for Aim 2. Children were nested within the RCH. Interaction terms were included for overall fidelity with change in the awareness, reaction, and involvement of other RCH staff with the Wellness Team.

A two-level path analysis with a robust maximum likelihood estimator was used to explore the relationships between awareness, reaction, involvement, intervention fidelity, and child FV intake. With this analytic procedure, the hierarchical structure of the data can be taken into full consideration. As awareness, reaction, and involvement were interrelated in bivariate analyses (r = 0.18–0.57), covariations among these variables were included in the model. Final models were adjusted for age, sex, race, and BMI.

The direct and indirect effects of each of the three support items on FV intake were assessed on the between-person level by first testing whether awareness, reaction, and involvement are directly related to changes in FV intake, and then testing whether awareness, reaction, and involvement are indirectly related to changes in FV intake through intervention implementation fidelity.

Model estimates indicated suppression effects and model justification issues likely due to small RCH sample size. Therefore, an alternative model was specified to achieve a more parsimonious and justified model. As a result, non-significant paths were eliminated from the model with no significant change in model fit. The critical significance level for standardized path coefficients was set conventionally at α = 0.05 and statistical fit criteria were used to assess model fit. Data were modeled using MPlus Version 7.11 (Muthen and Muthen 2007).

Results

The mean implementation fidelity scores were 1.66 (± 0.29) for the intervention condition and 1.29 (± 0.24) for the control group during the delayed intervention period. Median and range scores for implementation and organizational support at baseline are shown in Table 1. There were no differences in organizational awareness, reaction, or involvement regarding the intervention at baseline (P > 0.05). However, implementation was greater in the intervention RCHs compared to the control (P < 0.05).

Table 1 Organizational characteristics (baseline) and implementation scores
Full size table

When adjusted for group assignment, race, sex, age, BMI, and years a child had been at the RCH, there was no significant difference in daily FV intake by group assignment at the 2004 baseline assessment (Fig. 1; intervention = 2.27 vs control = 2.12, P = 0.755). There was a significant increase in daily FV intake (intervention = 3.20 vs control = 3.44) within all RCHs from 2004 to 2006 (P = 0.022 for the intervention group, P = 0.015 for the control group). This increase was maintained in both conditions from 2006 to 2008. There was no significant difference in FV intake between intervention and control RCHs at the end of the intervention (P = 0.620) in the main effects model (i.e., no interaction effect of implementation fidelity).

Fig. 1
figure 1

Comparison of intervention and control groups on child fruit and vegetable intake (2004–2008)

Full size image

There was no evidence of organizational support moderating the relationship between implementation fidelity and FV intake as all interaction terms were non-significant (data not shown). Change in awareness, involvement, and reaction were significant predictors of implementation fidelity in the path analysis (Fig. 2). Increases in organizational support resulted in greater overall fidelity at the RCH (P < 0.05) independent of condition. FV intake of participants increased with greater levels of implementation. Because of the nesting of children within RCH, there was insufficient power to test mediation of all three organizational support variables simultaneously. Figure 2 illustrates the test of mediation with organizational awareness of the intervention. Overall implementation fidelity partially mediated the relationship between organizational awareness with FV intake of participants. There was no partial or full mediation for the relationship of organizational involvement with FV intake of participants, nor for organizational reaction.

Fig. 2
figure 2

Final two-level path model of associations between support, implementation, and fruit and vegetable intake. All reported path coefficients are standardized estimates. Model adjusted for age, sex, race, BMI, where BMI and age are set to covary. Non-significant path coefficients were trimmed from the model with no significant reduction in model fit. *P < 0.05

Full size image

Discussion

Children in both the intervention and control groups reported an increase in FV intake between 2004 and 2006 and these increases were maintained through 2008. However, the results were not different by condition. ENRICH process evaluation revealed a substantial secular trend, implementation of the Child Nutrition and WIC Reauthorization Act of 2004, which required participating organizations to develop wellness polices that include nutrition education goals and guidelines for food provided on site (United States House of Representatives, 2004). Policies were required to be in place by July 1, 2006, coinciding with the intervention period. RCHs that participated in the National School Breakfast and School Lunch programs (65% of intervention and control homes) were required to follow this legislation. In addition to the federal policy, participating RCHs were at a high level of readiness for change. Data collected through the formative research indicated that at many RCHs’ initiatives were in place already, suggesting the possibility that RCHs assigned to the delayed intervention group may have moved forward on their own (e.g., Hawthorne Effect) during the control part of the study design. Although there were no differences in FV intake between the early and delayed intervention groups, changes in RCH environments mediated partially through implementation fidelity as assessed by ENRICH may be linked to healthier eating.

The findings suggest that organizational support for a structural intervention is associated with increased fidelity (Fig. 2). When Wellness Team members, supervisors, and the RCH CEO were reported to be supportive of the intervention (via involvement in and/or reaction to the intervention), more environmental changes were made to encourage eating FV. This is similar to studies in school settings where perceptions of principal support (Wanless et al. 2013) and organizational support (McBride 2000) were salient for greater fidelity.

Interestingly, organizational awareness had a direct effect on FV intake but in an unexpected direction. The coefficient was negative, suggesting that as awareness by RCH staff increased, child FV intake decreased. Perhaps this is related to rapid turnover of staff and children in the RCH environments. An alternative explanation may be that in RCH where environmental changes were more difficult to accomplish, the Wellness Team staff was putting in more effort, thereby increasing awareness without a concomitant change in FV intake. Additionally, as the constructs for organizational support were based on perceptions of awareness among different levels of staff, the level of awareness of ENRICH may vary by the referent in question (i.e., other staff, direct supervisor, and RCH CEO).

Due to the small number of RCH in the study, the mediation model could not be tested for awareness, reaction, and involvement simultaneously. Awareness alone does not necessarily indicate organizational support, but may be a precursor to greater involvement and support, as in theories of organizational change management (Varkey and Antonio 2010). In addition to the limited RCH sample size, participation was limited among the children living in the RCHs due to challenges obtaining consent from guardians or caseworkers not located at the RCH. However, little research includes or focuses on RCHs as a setting to reach youth. More studies are needed to better understand the complex social and built environment influences on health behaviors among children living in RCH. The measurement of organizational reaction to health promotion interventions warrants further examination based on the low alpha in this study. Finally, the impact of the federal policy supporting enhanced school-based nutrition policies may explain, in part, the lack of differences in the intervention and control (i.e., delayed intervention) groups in FV intake. Because this policy overlapped with the goals of the ENRICH intervention, it was difficult to tease out the impact of the federal policy change. However, as a structural intervention, ENRICH could be used as a model for implementing wellness policies and improving health outcomes in other youth settings.

Implications

The findings from this study are aligned with the evidence for organizational support in schools and worksites. Changing the organizational environment may be important for facilitating FV consumption. RCHs that have policies and environments supportive of wellness may improve dietary behaviors in youth. In the present as in other studies, increasing organizational support can improve program implementation. Finally, the results of the current study highlight the need for comprehensive process evaluation that includes assessments of the organizational setting and context in both intervention and control conditions. Tracking changes in the built and social environments for structural or other interventions, including possible effects of secular trends, provides insight as to why behavior change was or was not observed.

Conclusion

Adults and children regularly interact within a variety of organizational contexts. Understanding the ways in which organizational supports can enhance or detract from health promotion program implementation fidelity is important for improving the effectiveness of interventions. Fostering organizational support may improve implementation of interventions. The field of health promotion is engaging with systems science to foster healthier environments. The better equipped researchers are to recognize opportunities for organizational support, the more we increase our likelihood of successful interventions.