Introduction

Schools are the most common setting for the delivery of mental health services to youth in the USA (Costello, He, Sampson, Kessler, & Merikangas, 2014; Farmer, Burns, Phillips, Angold, & Costello, 2003; Langer et al., 2015; Lyon, Ludwig, Vander Stoep, Gudmundsen, & McCauley, 2013; Merikangas et al., 2011). There is immense pressure for public schools to adopt a continuum of evidence-based practices (EBP), defined as those with the best research evidence, clinical expertise, and that cater to patient preference and culture (APA Presidential Task Force on Evidence-Based Practice, 2006; Cook & Odom, 2013), across universal, targeted, and intensive levels of services to improve the mental health of youth (Fixsen, Blase, Metz, &Van Dyke, 2013; Odom, Cox, Brock, & National Professional Development Center on ASD, 2013). However, successful adoption, delivery, and sustainment of EBP in schools are fraught with challenges (Owens et al., 2014). When EBP are adopted in schools, only 25–50% are implemented with fidelity (i.e., implemented as intended) (Cook & Odom, 2013; Gottfredson & Gottfredson, 2001). Delivery of EBP with poor fidelity is unlikely to be effective in changing youth outcomes (Durlak & Dupre, 2008). This is a critical issue that results in significant wasted resources and weakens the potential of schools to promote youth’s mental health outcomes.

Recent research has examined the role of organizational constructs in the successful use of EBP in other service sectors such as specialty mental health and child welfare (e.g., Aarons et al., 2012; Beidas et al., 2013, 2014, 2016a; Bonham, Willging, Sommerfeld, & Aarons, 2014; Ehrhart, Aarons, & Farahnak, 2015; Powell et al., 2017), which also have relevance to schools (Forman et al., 2013; Hoagwood & Johnson, 2003; Langley, Nadeem, Kataoka, Stein, & Jaycox, 2010; Owens et al., 2014). Organizational constructs may play a critical role in the implementation of school-based services that maximize school mental health. Lyon et al. (2018) used the Exploration, Preparation, Implementation, Sustainment framework (EPIS; Aarons, Hurlburt, & Horwitz, 2011), a four-phase, prospective implementation framework that delineates the nested structure of outer (i.e., larger system-level) and inner (i.e., building level) contextual constructs that drive EBP implementation efforts, to define the organizational implementation context (OIC). The OIC represents malleable constructs specific to the inner context (i.e., microsystemic factors associated with a given school building) that influence successful EBP implementation in schools (Lyon et al., 2018). The OIC represents the immediate setting in which implementation takes place. As such, OIC constructs are most proximal to and likely to have an influence on implementer behavior than other factors outside that are farther removed from the place where implementation happens (Jacobs, Weiner, & Bunger, 2014; Malloy et al., 2015). The OIC captures three core organizational constructs: implementation leadership, implementation climate, and implementation citizenship behavior. These OIC constructs have been less frequently studied in schools but are likely to serve as proximal contextual indicators of behavior change among school-based practitioners (e.g., teachers, school counselors, licensed school-based mental health provider) who are responsible for the delivery of different services within a multi-tiered continuum of care (Bruns et al., 2016).

Implementation leadership refers to specific behaviors that leaders perform to support the implementation of EBP (Aarons, Ehrhart, & Farahnak, 2014a). Prior research has identified five dimensions of implementation leadership that combine to influence implementation outcomes in a given organizational context: organizational leaders’ being knowledgeable about the identified EBP, supportive behaviors directed toward frontline providers, proactive anticipation and problem solving regarding barriers that are likely to arise during the implementation process, perseverance with staying the course with implementation despite barriers that may arise, and availability to support and troubleshoot issues with frontline providers (Aarons et al., 2014a; Ehrhart et al., 2018). Implementation leadership can operate as a critical construct of the OIC that establishes a specific climate in a school that is conducive to EBP adoption, delivery, and sustainment across multiple tiers (Aarons, Ehrhart, Farahnak, & Sklar, 2014b; Aarons, Ehrhart, Torres, Finn, & Beidas, 2017).

Implementation climate is more specific than general school climate. While school climate reflects how individuals within schools perceive and ultimately describe the environment in their school, implementation climate reflects staff’s shared perceptions of the policies, practices, and procedures supporting EBP implementation, as well as the kinds of behaviors that are expected, supported, and rewarded as part of the implementation process (Ehrhart, Aarons, & Farahnak, 2014). Ehrhart et al. (2014) identified six dimensions of implementation climate including the organization’s: Focus on EBP, Educational Support for EBP, Recognition for EBP, Rewards for EBP, Employee Selection for EBP, and Selection for Openness. Organizations with low levels of implementation climate fail to demonstrate that EBP implementation is a valued endeavor as there is limited focus on EBP, support provided, and/or forms of recognition and acknowledgment for staff who invest in EBP implementation. Although the two constructs are distinct, implementation climate is primarily driven by implementation leadership and is supported by specific leaders’ behaviors that communicate those norms and expectations and allocate necessary resources (e.g., protected time, materials, money) to demonstrate the organization’s values within a given context (Aarons et al., 2014a). Existing research suggests that such focused or strategic climates are more related to specific outcomes rather than more molar or general measures (Schneider, Ehrhart, & Macey, 2013). Implementation climate may be a critical construct to examine in schools to help support the use of EBP (Locke et al., 2016).

Lastly, organizational citizenship behavior is exhibited when employees go “above and beyond” their core job aspects or standard “call of duty” to further the mission of the organization (Organ, Podsakoff, & MacKenzie, 2005). Applying this concept to the goal of EBP adoption and sustainment, implementation citizenship behaviors are those that demonstrate a commitment to EBP within the organization whereby individuals strive to keep informed about the EBP and offer support to their colleagues who are attempting to deliver the EBP with fidelity (Ehrhart et al., 2015). Ehrhart et al. (2015) posit two dimensions of implementation citizenship including: Helping Others and Keeping Informed. Implementation citizenship behaviors serve as a hypothesized mediator by which implementation leadership and implementation climate exert their influence on implementation success (Ehrhart, Aarons, Torres, Finn, & Roesch, 2016). Implementation citizenship captures specific behavior changes in school staff (e.g., helping a colleague with EBP implementation) and may mediate the influence of implementation leadership and implementation climate on successful EBP use.

Application to the School Context

Implementation research in other service sectors is generally more advanced than in schools (Sanetti, Knight, Cochrane, & Minster, in preparation). The multidisciplinary field of implementation science creates opportunities for researchers and practitioners to take advantage of existing findings and products from other service sectors, like health care, specialty mental health, and child welfare, by adapting and examining the application and generalizability of prior findings in novel contexts such as schools. For example, the ways in which the OIC constructs operate in the child welfare sector may look and operate differently in schools and there are few studies investigating implementation leadership, climate, and citizenship behavior within the context of EBP implementation in schools (Locke et al., 2016; Lyon et al., 2018). The absence of research in this area may be due to unknown relevance and appropriateness of OIC constructs to the education sector without systematic consideration of the school context (Lyon et al., 2018). It is possible that the manifestation of implementation leadership, climate, and citizenship in schools differs from the ways in which it is measured in other service contexts (e.g., hospitals, community mental health agencies). In addition, each of the OIC constructs may be perceived differently by individuals in different school roles (central office administrators, principals, teachers). Understanding the perspectives of different stakeholders can provide a more nuanced understanding of OIC constructs (Beidas et al., 2016b) in schools and may point to the need for a shift in thinking with regard to EBP implementation. Lastly, the boundaries of the OIC constructs may be more or less expansive in schools than in other settings in which they were initially conceptualized, resulting in the addition or deletion of specific subconstructs. This study may help guide future EBP implementation in schools.

Purpose of this Study

The current study occurred in the context of a larger sponsored project to adapt a suite of measures capturing OIC constructs for use in schools to support data-driven continuous improvement of evidence-based universal supports targeting student social, emotional, and behavioral outcomes. The purpose of this study was to use qualitative methods to examine how each of the OIC constructs (implementation leadership, implementation climate, and implementation citizenship behavior) is conceptualized in schools to determine whether the OIC constructs have different meaning and application than other service sectors in which they have been assessed (e.g., community mental health and child welfare), as well as generate potential relevant subconstructs that fall under the three broader OIC constructs. Given that school-based mental health involves the delivery of a continuum of services across universal, targeted, and intensive levels of care (Bruns et al., 2016), this study included different groups of educational stakeholders (i.e., district administrators, principals, teachers) involved in varying ways with implementation. For example, central administrators often manage and track implementation across schools, principals oversee and support implementation within their own schools, and teachers are the primary implementers of universal supports or collaborators on more intensive mental health interventions. The inclusion of different stakeholder groups also allowed us to examine whether perspectives of the OIC constructs varied as a function of group.

Methods

Setting and Participants

This study occurred as part of a larger federally funded project examining the iterative adaptation and validation of specific measures capturing key constructs of the school OIC. The university institutional review board and each participating school district approved the study. All participants provided informed consent prior to their participation. We first contacted a central administrator from each of the two partnering school districts. Next, the central administrator provided the names of 6–8 staff for each of three stakeholder groups: central administrators, elementary school principals, and elementary school teachers. Our recruitment process resulted in a total of 37 individuals (16 central administrators, 10 elementary school principals, and 11 elementary school teachers) from two relatively large school districts in the Northwestern USA. The two school districts are socioeconomically (approximately 26% of students in these districts qualify for free and reduced lunch) and racially/ethnically diverse (41.3% White; 26.8% Asian; 11.6% Hispanic/Latino; 9.6% Multiethnic; 8.7% African American; 0.3% American Indian or Alaskan Native; 0.2% Native Hawaiian). Together, these two school districts have 91 elementary schools. The sample was predominantly female (n = 29, 78.38%) with an average age range of 35–44 years old. Age bands were collected rather than exact ages to accommodate our participant population who felt more comfortable reporting ranges. The ethnic backgrounds of participants were as follows: 75.68% white, 10.81% multiracial/multiethnic, 5.40% Asian, 2.70% African American, 2.70% American Indian/Alaskan Native, and 2.70% Native Hawaiian or Other Pacific Islander. Their highest educational attainment was as follows: 8.11% had a doctoral degree, 86.49% had a master’s degree, and 5.40% had a bachelor’s degree. See Table 1 for demographic information.

Table 1 Demographics
Full size table

Procedures

Our theoretical approach uses the EPIS framework to develop a focus group protocol (see below and the Appendix) to elicit information about the relevance and appropriateness of measuring implementation leadership, implementation climate, and implementation citizenship in schools (Aarons et al., 2011). Focus groups were selected instead of other qualitative methods in order to allow synergistic discussion when individuals in the same focus group could elaborate on points articulated by other participants to provide a deeper understanding of how the OIC constructs manifest in schools. This approach was appropriate given the time and financial constraints of the grant award that did not permit the use of individual interviews. Focus groups were held at the central office of each respective school district to provide a convenient and accessible location for participants.

Separate focus groups were held for each stakeholder type (central office administrators, principals, and teachers) to remove obstacles to participant engagement (potential power differentials between stakeholders who may be in supervisory/managerial roles) as well as to allow for comparisons across stakeholder groups. The size of each focus group (6–9 participants) was consistent with recommended procedures for thematic saturation (Guest, Bunce, & Johnson, 2006), with the exception of one principal focus group which only had four participants because two had last minute conflicts that prevented them from attending the session. Each focus group session began with establishing norms and expectations for the session, introductions, and an overview of constructs (e.g., implementation leadership, climate, citizenship behavior) that were to inform the development of a set of school-based measures (Lyon et al., 2018). Prior to the focus group discussions, facilitators defined EBP and implementation science for all participants. Participants had varying levels of understanding and experience with EBP implementation. The focus group moderators situated the discussion of EBP with a focus on Tier 1 or universal programs that promote students’ social, emotional, or behavioral functioning, which was the focus of the larger study (Lyon et al., 2018). Participants also were provided an overview of each construct (definitions, how each construct is described in the literature) and asked to review existing measures (e.g., Implementation Leadership Scale, Implementation Climate Scale, and Implementation Citizenship Behavior Scale). Discussion of each of the OIC constructs ensued after measure review, with specific probing questions designed to elicit feedback from the participants about each construct’s manifestation in the school context. Each focus group was audio- and video-recorded and lasted 90–120 min. Participants were compensated $150 for their participation.

Measures

We developed a systematic and comprehensive focus group protocol with questions that elicited open-ended responses from participants about: (1) implementation leadership (e.g., “What steps can leaders take to support their staff in successful implementation?”); (2) implementation climate (e.g., “What specific aspects of implementation climate come to mind when you think about schools in which the adoption and implementation of evidence-based practices are high priorities for leaders and staff?”); and (3) implementation citizenship behavior (e.g., “What is it about an employee in a school that allows him/her to support colleagues in implementing evidence-based practices effectively?”).

Data Analysis

Focus groups were conducted by Lyon et al. (2018). Focus groups were transcribed and uploaded to NVivo QSR 10 for data management. The EPIS model and OIC guided the development of the coding scheme. The coding scheme was developed using a rigorous, systematic, transparent, and iterative approach using the following steps. First, Lyon et al. (2018) independently coded two initial transcripts line-by-line to identify recurring codes. Second, they met as a group to discuss recurring codes and developed a codebook using an integrated approach to coding as certain codes were conceptualized during the focus group protocol development (i.e., deductive approach) and other codes were developed through a close reading of the two transcripts (i.e., inductive approach; Bradley, Curry, & Devers, 2007; Neale, 2016). Next, coders and principal investigators met to discuss and select common codes interpreted from the transcripts. The group collectively determined which codes were incorporated into the final codebook. Then, operational definitions of each code were documented as well as examples of the code from the data, as well as when to use and not use the code. The coding scheme was applied to the data to produce a descriptive analysis of each code; the coding scheme was then refined throughout the data analytic process (Bradley et al., 2007). A table of codes and definitions is provided in the Appendix. Lyon et al. (2018) coded all data and overlapped on 20% of randomly selected transcripts to determine inter-rater reliability. They met together on a weekly basis to discuss, clarify, verify, and compare emerging codes to ensure consensus. Agreement was calculated based on the number of words agreed upon; agreement between raters was excellent (percent agreement = 95.05% on parent codes and 98.60% on subcodes).

Results

Participants’ conceptualizations of each of the constructs indicated unique elements of the school context that may be important for successful implementation of EBP to support school mental health. However, there also were many similarities with the definitions and item content associated with the original development of the OIC measures in the implementation science literature. Overall, central administrators and principals were broader in their descriptions of how the constructs manifest in schools, whereas teachers were more likely to focus on the specifics of EBP implementation. See Table 2 for a list of codes and definitions and Table 3 for a list of OIC constructs, definitions, existing and new domains. Below we address each construct, noting which emerged as “new” and which previously “existed” in the original measures.

Table 2 Definitions of codes
Full size table
Table 3 Original OIC construct definitions, existing domains, and added domains
Full size table

New Dimensions

Several new dimensions emerged from each of the OIC constructs that are unique to the school context. First, Distributed Leadership was identified within the implementation leadership construct. Participants from all three stakeholder groups discussed Distributed Leadership across multiple individuals (e.g., teacher leaders, career ladder teachers), teams (school psychologists, counselors, teacher leaders, teacher mentors/coaches), and levels (schools and central office) to support EBP implementation. Overall, many participants from each of the three stakeholder groups challenged the notion that leadership in schools is centralized at the school administrator level and highlighted the importance of a team-based leadership approach. One principal said, “There are a lot of different leaders in the building… the principal might not know everything about EBP, so we rely on our experts in the building to know those answers.” A central administrator highlighted that “the building leadership team develops a plan [for EBP implementation] versus the school administrator [often] establishes clear school standards and accountability measures for implementation.” Although the principal often is seen as the primary leader in a school, the general sentiment across all three stakeholder groups was that leadership teams (which comprise several stakeholders in school settings) share the responsibilities of supporting EBP implementation.

Second, Existing Supports to Deliver EBP was a new dimension of implementation climate that emerged across all participants that captured how existing school structures or resources could be incorporated or repurposed to support EBP implementation and build capacity across levels (individual teacher/staff, school and central office). All participants mentioned using existing school teams (i.e., guidance teams, school improvement planning teams, career ladder teachers, grade-level teams, professional learning communities) that are naturally embedded in schools’ existing infrastructure to play a role in the adoption, implementation, and sustainment of EBP in schools. One central administrator commented that “there are routines or structures established within the school [such as] coordinated professional learning communities or planning time…that communicates with people and [trains] new staff to make [EBP implementation] more feasible” in schools. The reliance on existing school teams may facilitate EBP use and champion implementation efforts forward on multiple levels. Another aspect of Existing Supports endorsed by all stakeholder groups was the use of peer-to-peer relationships and informal social interactions among teachers/staff to support EBP implementation efforts by reducing feelings of isolation that are common in schools.

Third, Prioritization of EBP also was a new dimension of implementation climate created to capture district and school priorities, policies and procedures, and their alignment with EBP implementation. Participants varied in their description of Prioritization of EBP. Some central administrators remarked that Prioritization of EBP often is based on district-level support and resources (i.e., funding) or district priorities, even though there is a “disconnect between what the district prioritizes and what actually happens [in practice].” Principals elaborated on the challenge of competing priorities in academic versus social-emotional EBP that teachers/staff manage that often leads to EBP fatigue. One principal said that “we are so consumed with a lot of other academic frameworks that we would like to get to the social-emotional learning standards…but, they are not a prominent feature.” Both principals and teachers expressed the need for central office and school leadership to de-prioritize competing EBP and other initiatives more broadly and suggested that “clearly defined” school priorities around EBP and initiatives is essential to gain buy-in from teachers/staff.

Fourth, Information Sharing was a new dimension that emerged as part of implementation citizenship behavior. The majority of principals and teachers identified Information Sharing in their discussion of implementation citizenship behavior where school personnel actively shared knowledge (e.g., expertise, tips, strategies) with their colleagues, peers, and broader community (parents). Principals also discussed that it was “above and beyond” behavior to “put them[selves] out there” to “share knowledge or [their] expertise with others within [their] building as well as within [their] district or beyond.” Observation/Feedback also was a new dimension of implementation citizenship. Most participants described this construct as willingness to open up their practices for observation and feedback as well as provide opportunities for educational site visits from other schools/districts to observe their practices. One central administrator noted that “having [staff] that [implement EBP well] model those practices for other staff and colleagues” is critical to this construct.

Existing Dimensions

Implementation Leadership

Comments from some central administrators and principals aligned with Aarons et al. (2014a) definition of proactive leadership. Many central administrators and principals described proactive leadership in terms of collecting multiple sources of data to identify growth and areas of improvement as well as track accountability. One central administrator indicated “using data to show where there’s growth and where there’s room for improvement” would be helpful in increasing EBP use. Some central administrators and principals described supportive leadership similarly to Aarons et al. (2014a) by saying that leaders were: (1) present and fully engaged in implementation efforts; and (2) able to provide resources (e.g., time, physical space, embedded coaching, behavioral observations, materials). One novel component described by central administrators and teachers was showing support by helping to “prioritize or de-prioritize” competing demands or initiatives in shared decision making given the EBP fatigue or overload that often exists in schools. Some teachers differed in their depiction of supportive leadership and expressed that supportive leadership entails a leader who: (1) is willing to learn about EBPs alongside teachers and implementation agents (e.g., “I would like to see administrator willingness to get involved in the learning with [teachers, where they] come to [the] classroom and do the strategy…”); (2) understands and has empathy for the implementation challenges that teachers face; and (3) is intentional about creating structures (e.g., check-ins with whole staff and individuals, troubleshooting) to ensure EBP implementation is prioritized.

Similar to Aarons et al. (2014a) definition, central administrators and teachers characterized knowledgeable leadership as providing “the why” to teachers and staff to increase understanding of the expectations, needs, and goals of EBP implementation. Many principals noted additional aspects of knowledgeable leadership including the leader’s ability to “consistently model EBP” and the ability to discern and use credible “evidence to support decision making to use EBP.” Perseverant leadership was infrequently discussed across the participants. Most principals described perseverant leadership as active involvement with teacher and school staff implementation efforts to create an environment where “EBP implementation carries on even in the absence of the principal.” Consistent with Ehrhart et al. (2018), central administrators and teachers discussed available leadership in terms of being open and present to provide help and feedback, while principals did not mention this construct. Teachers underscored the importance of an available leader as someone they can trust to have an “open door to bring up concerns” to enhance implementation or barriers to implementation. One teacher specifically noted that available leadership entails “physical availability… actually having [their] door open or being in the classroom and… emotional availability [where they are] willing to address something.”

Implementation Climate

Many principals mentioned having a “building focus” on EBP as well as setting “explicit goals around [adoption of] EBP” among staff carrying out implementation. Similar to Ehrhart et al. (2014), participants described educational supports for EBP in the context of professional development trainings (i.e., workshops, seminars, conferences) that are initiated and supported at the district level across all schools. One administrator mentioned, “there is an underlying assumption that everybody understands what an EBP is…[but she] never [uses the term] EBP because people would freak out…and assume that it is a major effort to implement.” Many teachers discussed the lack of educational support in terms of tangible materials. One teacher noted that she has “gone to training, [but] if [she] wants to implement [the EBP, she] would have to buy it or have the materials to be able to copy and paste all the stuff.” Additional aspects of Educational Support for EBP that many principals and teachers brought up included having: (1) access to experts in the field available to teachers and school staff; and (2) protected time to reflect, adapt, develop implementation plans, and integrate EBP into daily practice.

Recognition for EBP was infrequently discussed. The only component of the existing subscale that participants noted was the potential for promotion. One principal described “greater access to leadership opportunities” as compared to “traditional promotion” as recognition opportunities in schools. All participants agreed and stated that traditional Rewards for EBP (i.e., in the form of promotions or financial incentives) were not appropriate or feasible within the school context. Central administrators explained how “financial incentives [like] bonuses and raises are off the table…[due to] union contracts and involvement.” Furthermore, some principals specifically noted that financial incentives are “not a motivator” and “will divide [them] as a team” since “the heart of why [school staff] do what [they] do…” is to see “[their] kids succeed.” Interestingly, teachers articulated that they “get so much more [from] being compensated for time to plan” around EBP implementation, which would be the most favorable reward that they could receive over and above monetary incentives. Selection for EBP was mentioned twice and only among teachers. Teachers expressed the lack of understanding among school staff as to how new hires are selected. One teacher said, “Unless [school staff] are on the hiring committee, [they] do not know why a person is selected.” The reasons that some people are selected and hired into their roles appear to be largely unknown to teachers in participating schools. Selection for Openness was not discussed.

Implementation Citizenship Behavior

Participants’ conceptualization of Helping Others was similar to Ehrhart et al. (2015) definition with respect to responsibilities for EBP implementation and helping teach EBP implementation procedures; however, none of the participants described helping to ensure proper EBP implementation (monitoring fidelity). Most frequently across participants, Helping Others was described in the context of teamwork, “equally sharing the workload among staff” via formal (e.g., professional learning committees, grade group meetings, demonstration teachers) and informal methods (e.g., distributing responsibilities, mundane daily tasks). Keeping Informed was not frequently discussed. Only one component of the Keeping Informed subscale from Ehrhart et al. (2015) definition, latest news regarding EBP, was discussed among teachers. Central administrators described “attending trainings [and professional development] as a key indicator” of Keeping Informed as well as keeping up with the “latest news [such as] reading online sources” wherever available during personal off-work times. However, district- or school-wide communication related to EBP and changes in EBP policies and procedures were not mentioned.

Discussion

Adaptation and application of existing constructs and corresponding measures of the OIC to the school setting provide promising opportunities to advance school-based implementation science and practice. This study extends research led by Aarons et al. (2014a) and Ehrhart et al. (2014, 2015, 2018) to articulate key aspects of the OIC of schools that relate to EBP implementation success in two local school districts in the Northwestern USA. See Fig. 1. Specifically, this study engaged key education stakeholders (central administrators, principals, and teachers) to gather information regarding their perceptions of the appropriateness and conceptual boundaries of the OIC constructs (implementation leadership, climate, and citizenship behavior) in schools. Three general findings emerged from our focus groups. First, we found critical differences unique to the education sector and several similarities with other service sectors (e.g., community mental health and child welfare) in how OIC constructs manifest, which are further discussed below. Second, all stakeholder groups indicated that greater attention to the OIC constructs offers a promising approach to improving youth mental health outcomes through the adoption and delivery of higher quality school-based mental health services and supports. Last, the OIC constructs appear to have potential utility across each stage of the implementation process (exploration, preparation, implementation and sustainment; Aarons et al., 2011). Based on the data gleaned from this study, we offer recommendations to researchers and community stakeholders who may be interested in examining these organizational constructs in schools.

Fig. 1
figure 1

School organizational implementation context (OIC) concept map. The light gray bars depict new dimensions of the OIC constructs in schools

Full size image

We note that many dimensions of the OIC were not frequently discussed that may play a critical role in successful EBP implementation in schools. For example, perseverant leadership may be one aspect of implementation leadership that warrants more attention in schools since implementation is a long and arduous process that often takes 2–5 years or more for full implementation to occur (Fixsen, Naoom, Blasé, Friedman, & Wallace, 2005). A lack of perseverant leadership may contribute to the number of EBP implementation efforts that fail in schools (Gottfredson & Gottfredson, 2001), and the frequently encountered “flavor of the month” phenomenon in which organizations get stuck in a cycle of taking on and abandoning new programs and practices (Basch, Sliepcevich, Gold, Duncan, & Kolbe, 1985). Schools should carefully consider the ways to problem solve implementation challenges prior to decommissioning an EBP, which may involve the de-implementation of other ineffective programs and practices (Wang, Maciejewski, Helfrich, & Weiner, 2017). In addition, participants in the current study augmented implementation leadership with an additional dimension: Distributed Leadership, where leadership is shared across multiple individuals and roles in a school as well as the central office (Angelle, 2010). This dimension captures shared leadership responsibilities that are held across stakeholders in schools. Although there also was recognition among participants that principals tend to be the ultimate arbiters of most school-related issues, there may be various leaders that spearhead different tasks or content areas in schools that embody middle leaders or middle managers, and implementation efforts ought to consider the critical role that multiple leaders may play in schools (Priestland & Hanig, 2005; Birken et al., 2015).

Our results suggest that implementation climate may manifest somewhat differently in schools. While some components (Focus on EBP; Educational Support for EBP) were similarly conceptualized to more traditional allied healthcare delivery settings, other aspects of implementation climate differed. First, it was clear that selection for EBP and selection for openness may not be feasible components of implementation climate measurement in schools, as some stakeholders at the school building level often do not have influence in the hiring process to select staff with a proclivity for use of EBP. It is concerning that teachers described the selection and hiring process as lacking transparency. Our participants often indicated they were not aware of the district standards for hiring new employees, which may indicate that hiring standards are not readily available in the districts in which this work was conducted, or they do not exist. Without transparent processes, it is more difficult to make changes that might support the recruitment and selection of individuals with more openness, knowledge, skills, and abilities related to the use of EBP. Second, our findings suggest that the reward and recognition components of implementation climate become increasingly blurred since there is universal agreement that financial rewards are inappropriate and unfeasible in schools (Lyon et al., 2018) or, that they may receive less emphasis compared to the potential benefits of EBPs for students. These results are consistent with Lyon et al. (2018), who quantitatively examined implementation climate in schools and found that the factor structure of most implementation climate subscales was upheld but there was less evidence supporting the rewards subscale. However, consistent with one item in that subscale, our stakeholders posited non-financial incentives—such as the provision of protected time for planning or additional preparatory periods—as potential recognition or reward for EBP use, which central offices and schools may consider to develop positive implementation climates. Third, two new dimensions (Existing Supports to Deliver EBP and Prioritization of EBP) emerged that also may capture important aspects of implementation climate in schools. Given the financial constraints that many schools face, stakeholders discussed the importance of capitalizing on existing school structures and supports and leveraging existing structures (teams, personnel) to maximize limited resources to facilitate EBP implementation. This may help mitigate resource-related barriers to implementation. Similarly, stakeholders identified Prioritization and De-Prioritization of EBP and other initiatives as an important component of implementation climate. Schools often experience significant EBP fatigue and prioritizing certain implementation efforts may increase the likelihood of successful EBP adoption, use, and sustainment. Lastly, OIC constructs may be malleable determinants in schools that act as mechanisms through which organizationally focused implementation strategies may impact implementation outcomes. This conceptualization mirrors the direction of increasing implementation research focused on developing a more robust understanding of how implementation strategies exert their effects (Lewis et al., 2018). With proper implementation strategies deployed, it may be possible to improve these aspects of implementation climate and improve EBP use in schools (Cook, Lyon, Locke, Waltz, & Powell, submitted; Lyon, Cook, Locke, Powell, & Waltz, submitted).

Similar to implementation climate, our results suggest that aspects of implementation citizenship behavior (Helping Others and Keeping Informed) were comparably conceptualized in schools; however, there also were unique differences that stakeholders highlighted for implementation citizenship behavior in schools. The Helping Others and Keeping Informed dimensions were mostly present in schools with three notable differences—none of the stakeholders discussed: (1) ensuring proper EBP implementation; (2) district- or school-wide communication related to EBP; or (3) changes in EBP policies and procedures, which suggests these behaviors are rare, not salient, or nonexistent in school settings. These may be important behaviors to cultivate to support EBP implementation in schools. In addition, two new subconstructs of implementation citizenship emerged: Information Sharing and Observation/Feedback. Stakeholders highlighted the necessity of Information Sharing with their colleagues, particularly sharing their expertise with others in their building and beyond. Interestingly, stakeholders also noted the value of observation and feedback—both opportunities to provide and receive Observation/Feedback on their performance. These additions to implementation citizenship may be critical practices in the education sector. Strategies to improve opportunities for collaboration and feedback may help strengthen implementation citizenship in schools. Moreover, identifying and supporting key opinion leaders (Atkins et al., 2008) and EBP champions (Aitken et al., 2011) may reflect social strategies to influence implementation citizenship behaviors that facilitate EBP uptake, delivery, and sustainment. However, the extant literature offers little guidance on strategies to promote implementation citizenship in schools.

The results of our study also were consistent with the implementation leadership dimensions (proactive, supportive, knowledgeable, perseverant, and available) found in the broader implementation science literature (Aarons et al., 2014a; Ehrhart et al., 2018). However, it appears that some dimensions were more salient and prevalent (supportive) in schools than others (perseverant). Participants noted that supportive leadership encompassed the provision of tangible resources (materials) as well as emotional support for EBP use. Interestingly, participants also discussed Prioritization and De-Prioritization of EBP and other school initiatives to prevent EBP fatigue as supportive leadership. This is an important point to consider as schools often face barriers to implementation that include inadequate time, energy, effort, and resources for implementation (Forman, Olin, Hoagwood, Crowe, & Saka, 2009; Locke et al., 2015). Indeed, in a recent school-focused adaptation of the Expert Recommendations for Implementing Change (ERIC) compilation of implementation strategies in health care (Powell et al., 2015), Cook et al. (submitted) added “pruning competing initiatives” to address exactly this issue.

Limitations and Future Directions

While this study explored the qualitative nuances of implementation leadership, climate, and citizenship behavior in schools, several limitations are noted. First, these results should be interpreted with caution as the data only were gathered in two predominantly white school districts in the Northwestern USA, which may limit the geographic generalizability of the results to areas with similar racial/ethnic diversity and/or socioeconomic status as well as school characteristics. Second, the teachers who participated were not self-nominated, but were nominated by the district’s central administrators. Third, the sample was predominantly female, which may limit the interpretation of the results. Fourth, the current project was focused primarily on Tier 1 universal supports. The OIC constructs investigated may manifest differently for more intensive social, emotional, or behavioral supports. Lastly, this study only focused on the OIC constructs, and no individual-level constructs were explored, which also is important for EBP implementation (Becker-Haimes et al., 2017; Feuerstein et al., 2017).

Conclusion

Findings from this study support the assertion that the three OIC constructs (implementation leadership, climate, and citizenship) are relevant and applicable to the school context. Results also demonstrate the utility of qualitative methods to inform revisions to existing constructs from other service settings (e.g., health care, child welfare, juvenile justice) with the goal of facilitating application in novel settings, such as schools. As noted earlier, the OIC constructs also have been codified in a set of established organizational evaluation measures (Lyon et al., 2018), which may be further revised based on the information collected herein. Next steps could include validation of the expanded versions of OIC constructs in the context of school-based implementation efforts. Use of qualitative methods to adapt existing measures is one way to avoid the “home-grown measure” phenomenon in which measures are rapidly created “in house to assess a construct in a particular study sample without engaging in proper test development procedures” (Martinez, Lewis, & Weiner, 2014, p. 4). This phenomenon has recently plagued implementation science and diminished the comparability of results across studies. Home-grown measures often are highly specified to the population in which the measure was used and have limited generalizability and utility for comparisons across studies (Martinez et al., 2014). Additionally, there is a need for research that carefully considers how the OIC constructs manifest across individuals at different levels (e.g., individual provider, school/organizational, and central office/district) and how to design specific implementation strategies that precisely target specific aspects of OIC constructs in schools.