The Centers for Disease Control (CDC) estimates that 1 in 59 children have an autism spectrum disorder (ASD; Baio et al. 2018). Long term outcomes for this population have been found to be poor (Eaves and Ho 2008; Orsmond et al. 2013; Roux et al. 2013; Shattuck et al. 2012). Based on the high expenditures of care and lost productivity for individuals with ASD and their caregivers, the estimated annual cost of care in the US is $268 billion with an increase to $461 billion by 2025 (Leigh and Du 2015). Evidence-based behavioral interventions (EBIs) show strong support targeting multiple outcome domains (e.g., academic, communication, social, mental health/behavioral) (National Autism Center 2015; Wong et al. 2015). Despite costly investments to develop and test ASD EBIs, they are not routinely delivered in community-based care (Brookman-Frazee et al. 2012, 2010; Stahmer et al. 2005).

Children with ASD are likely to access a variety of public service systems (Mandell et al. 2005; Simonoff et al. 2008). The two public service systems particularly important for serving school-age children with ASD are education and mental health (Brookman-Frazee et al. 2009). In response to quality gaps in these settings, there have been urgent calls for the development and testing of implementation strategies to facilitate successful uptake and sustained delivery of EBI (Dingfelder and Mandell 2011; Forman et al. 2013). To address this need, our research groups have used community-partnered approaches to adapt and implement behavioral EBIs for ASD specifically for delivery in routine care in these service systems (Brookman-Frazee et al. 2016; Dyson et al. 2019; Stahmer et al. 2016; Wood et al. 2015). AIM HI (“An Individualized Mental Health Intervention for ASD”; Brookman-Frazee and Drahota 2010) refers to a package of EBI strategies designed to reduce challenging behaviors in children served in mental health service programs. CPRT (“Classroom Pivotal Response Teaching”; Stahmer et al. 2012) refers to an EBI adapted for use in education settings to target social, communication, behavior, and academic skills. AIM HI and CPRT share common methods for developing, adapting, and testing interventions in varied community service settings (Wood et al. 2015). Consistent with the Aarons et al. (2011) Exploration, Preparation, Implementation, Sustainment (EPIS) implementation framework designed for public service sectors, these interventions were designed specifically for, and with, the “end users” of the intervention, with the goal of maximizing the “fit” between the intervention and service context (Garland et al. 2010; Palinkas et al. 2008; see Fig. 1).

Fig. 1
figure 1

Applying the exploration, preparation/adoption, implementation, sustainment (EPIS) conceptual model of implementation to ASD EBIs

Full size image

The development and testing of both AIM HI and CPRT intervention and training models included systematic collection of community stakeholder input. For example, we have obtained provider (teacher and therapist) perspectives on general use of, and attitudes toward, evidence-based practices in both school (Stahmer et al. 2005; Stahmer and Aarons 2009) and community mental health programs (Brookman-Frazee et al. 2009, 2010; Barnett et al. 2017). For both interventions, leader and provider input factored heavily in the development of intervention manuals and materials, training procedures and implementation processes (Stahmer et al. 2012; Brookman-Frazee et al. 2012; Stahmer et al. 2016). Additionally, we have gathered extensive provider perspectives on both AIM HI (Drahota et al. 2014) and CPRT (Stahmer et al. 2012; Suhrheinrich et al. 2013) through focus groups and interviews with providers participating in early pilot studies of the interventions.

The AIM HI and CPRT research groups recently concluded separate and concurrent, large scale community effectiveness trials of these interventions (NIMH R01MH094317 [AIM HI]; IES R324B070027 [CPRT]). Consistent with calls for hybrid effectiveness/implementation research (Curran et al. 2012), we systematically and prospectively collected data on implementation processes and outcomes from multiple stakeholder groups. For example, we examined provider perspectives on inner context factors influencing AIM HI delivery and characterizing adaptations to AIM HI (Chlebowski et al. 2018; Dyson et al. 2019). In both studies, we applied the EPIS implementation Model (Aarons et al. 2011) to frame implementation processes across studies because it was developed for public sector services contexts and integrates a multi-level framework across implementation phases. The EPIS framework is one of the most highly cited implementation frameworks and has been used to guide more than 49 unique research projects across 11 countries. Studies have focused on implementation of EBPs for a number of health and allied health issues (behavioral problems, mental health, substance use, HIV, Parkinson’s disease, teen pregnancy, child maltreatment, and workplace disability) (Moullin et al. 2019). EPIS integrates factors influencing implementation from both outer (e.g., service system) and inner (e.g., program/school, leader, provider) contexts and draws attention to interrelationships of stakeholders across these contexts. Another important consideration highlighted in the EPIS framework is the fit of a clinical or service intervention across levels including system (e.g., school systems), organization (e.g., community clinic, a given school), provider (e.g., clinician, special education teacher), and client (e.g., children and families). Refer to Fig. 2 for the conceptual framework applied to this data collection.

Fig. 2
figure 2

Applying the EPIS framework for measuring AIM HI and CPRT implementation process

Full size image

In the current study, we capitalized on the opportunity to coordinate prospective data collection on implementation process through the two hybrid effectiveness/implementation trials and expand on our prior work in two main ways. First, we expanded on our prior research conducted individually to facilitated characterizing similar and unique determinants of implementation across two important service systems for ASD. Second, consistent with the ethnographic-informed “periodic reflection” approach (Finley et al. 2018), we sought to complement our prior research collecting data from community stakeholders with periodic and real-time reflections from the project team on the dynamic implementation context. Periodic reflections offer an innovative and pragmatic approach for documenting implementation phenomena within the context of a multi-method implementation evaluation. In the current study, qualitative data collected from the project teams were used to complement intervention-specific qualitative and quantitative data collection from participant stakeholders (leaders, providers, caregivers). Thus, the primary purpose of the current study was to apply the EPIS framework to characterize common and unique factors influencing implementation process for AIM HI and CPRT in mental health and education service contexts.

Method

Procedures

Consistent with the ethnographically-informed periodic reflection approach to documenting implementation phenomena (Finley et al. 2018), the perceptions of project implementation teams were gathered using qualitative methods. Multi-person interviews [study-specific multi-person interviews (PI + Project Manager) and focus groups (trainers)] were used to capture information generated by the interaction between team members. Refer to Table 1 for a summary of data collection sources.

Table 1 Data sources by type and study
Full size table

Intervention Developer (PI)/Project Manager Interviews

An independent researcher not connected with either study, but with experience conducting qualitative interviews and lead author of the EPIS (GA), conducted a total of nine semi-structured, multi-person qualitative interviews with PIs (n = 2 [one for each study]) and Study Managers (n = 3 [2 for AIM HI; 1 for CPRT]). Five interviews were conducted over the course of the AIM HI effectiveness study and four interviews were conducted over the course of the CPRT effectiveness study. The purpose of the intervention developer (PI)/manager interviews was to gather first-hand accounts of the implementation process across the Preparation/adoption, Implementation, and Sustainment phases of implementation according to the EPIS model and identify factors associated with implementation from the perspective of the intervention developers/ research team to inform improvements to future projects. Interviews were conducted 1–2 times per year during the periods of recruitment, active training/implementation, and sustainment data collection between 2013 and 2016. The interview guide was flexible such that the interviewer could follow up and explore relevant themes raised in the interviews.

Trainer Focus Groups

Two focus groups were conducted with the research teams’ expert trainers responsible for providing ongoing training to community providers (mental health therapists, school teachers) to inform future training efforts.

The first focus group was conducted with CPRT trainers only and was conducted by the study PI. The focus group guide was structured to gather CPRT trainers’ perspectives on barriers and facilitators to teacher use of CPRT and sustainment over time and to gather feedback training materials and processes. Consistent with the EPIS framework, questions targeted participants’ perceptions of how inner context factors -district, teacher, student, intervention, and training factors - impacted CPRT implementation and sustainment (e.g., “What teacher factors facilitated implementation of CPRT?” “What were facilitators of sustainment of CPRT at the school district level?”).

The second focus group was conducted with both AIM HI and CPRT trainers by an external researcher with knowledge of EPIS and qualitative interviewing training not involved in either study. The purpose of this focus group was to gather trainers’ perceptions of provider level inner context characteristics associated with training outcomes and identify potential provider level interventions to enhance training outcomes. Specifically, the guide assessed the following constructs: how and when trainers identified that trainees were experiences challenges during the training process and what specific strategies trainers used to facilitate success of these trainees. Trainers considered both their higher and lower performing trainees to help guide the discussion of individual characteristics that were associated with trainee challenges and success (e.g., “What is the first thing that you noticed about these trainees that made you begin to have a concern?” “What did you do differently, if anything, to tailor or modify your approach with these trainees?”).

Analysis plan

All semi-structured interviews and focus groups were audio recorded and transcribed for coding. A stepwise development of the coding system was employed starting with utilization of a codebook developed by investigators based on constructs of interest identified a priori (i.e. based on the EPIS constructs targeted in the interview guides). EPIS constructs were considered but not imposed unless supported by the data. Coding was conducted by two individuals experienced in qualitative analysis and not otherwise associated with the AIM HI or CPRT studies. Coders reviewed a subset of the transcribed interviews and individually developed and applied a series of codes informed by the EPIS framework to sections of the text to condense data into organized, analyzable units. These codes were then discussed and integrated to develop a codebook. The codebook contained definitions of codes, guidelines for use, and examples of representative quotes appropriate for inclusion in the category. Each interview transcript was then independently coded and discrepancies in assignments of codes were discussed and resolved amongst the coding team.

The NVivo (QSR International 2012) qualitative analysis software program was used to conduct thematic analysis through coding, development of categorical “nodes” consisting of related units of text, and aggregation of codes through the process of review and comparison in order to identify emergent themes and to ensure systematic analysis of coded data (Seale and Silverman 1997). This process was first completed separately for each service context [mental health (MH) or education (ED)]. To facilitate comparison of relevant factors between services, the team identified those that were shared and unique to each system. An iterative approach of revisiting codes as questions and connections emerged guided the process of data collection and analysis (Berkowitz 1997). Peer debriefing between the authors allowed for detailed discussions of the data and interpretation of the emerging themes. Data objectivity was maintained through the involvement of coders and co-authors not directly involved in data collection and analysis.

Results

Table 2 summarizes salient themes by service system—Mental Health (MH) or Education (ED), highlighting influences that were shared across service and those unique to one or the other system.

Table 2 Themes regarding implementation influences across service systems
Full size table

Outer Context (System/Policy)

Shared Influences

Outer context factors such as system leader facilitation of agency/district and provider participation were particularly important across both systems, and lack of system leader involvement was considered a barrier. For example, MH system leaders facilitated agency participation through their existing relationships and communication mechanisms with provider agencies. This was perceived as particularly important when the research team was not already known to agency leaders. Similarly, in schools, an endorsement from the Special Education Local Plan Area (SELPA) director, on recruitment materials and distributed through SELPA communications facilitated enrollment by district level leaders.

Unique Influences

The dynamic system structure, including funding and administrative oversight, was an outer context factor that only emerged in the context of MH services in the AIM HI study. Specifically, a change to state policy shifted responsibility of educationally related mental health services (i.e., mental health services included in students’ special education plans) from County Mental Health departments to individual school districts. This policy went into effect the same month as agency outreach and enrollment began for the AIM HI study and resulted in an immediate de-centralization of services and associated outreach. Districts took different approaches to provide these required mental health services including, initiating contracts with County-contracted provider agencies, establishing or growing an internal district MH program or integrating MH services within other special education services and these approaches evolved over the 4 years of agency recruitment. The decentralization and timing was a significant challenge as the research team was faced with establishing numerous additional contacts with individual school districts (for example, there are over 40 districts in one of the counties), refining eligibility (how to define a mental health “program”?) and monitoring changes in program structure that impacted eligibility (based on the number of clinicians within a district and the type of services provided). There were no policy changes highlighted as significantly impacting implementation in the CPRT study.

Inner Context Factors: (Program/District)

Shared Influences

Within the inner context, program/district leader engagement was considered a key influence across implementation in both MH and ED services, with higher leader motivation and engagement was considered an important facilitator and lower engagement was considered a barrier across both studies. Beyond obtaining initial leader approval to conduct study activities within their program or district, encouragement and facilitation (e.g., inviting the research team to recruit at standing meetings, presenting the EBI training opportunity as a part of existing training structures, enrollment and scheduling) offered important logistic support and was perceived to increase provider buy-in and motivation and reduce the burden of provider participation in MH services. Program supports surrounding time for staff training impacted preparation in ED services, with district or school leader willingness to support for time for training during work hours via provision of substitutes as both a facilitator and barrier.

Furthermore, leader involvement in the clinical and training process (e.g., attendance at training workshops or consultation or coaching sessions) was also perceived to facilitate implementation in both MH and ED settings. These findings indicate leadership buy-in and involvement as a key factor supporting implementation. Leaders who remained connected with the research team throughout the training and follow-up were perceived to have providers who remained more engaged in the EBI training process. Engaged leaders often led to requests for additional training, and provided easier access to providers over time.

Unique Influences

Several program level factors such as structure, program maturity (see dynamic system structure above), method of case assignment, administrative logistics and provider turnover were considered both facilitators and challenges in MH services, but were not considered to substantially impact implementation process in the ED system. Well-established, mature County-contracted programs that had structures in place for training and delivering mental health interventions were considered to facilitate training and consultation in the new EBI as these could be placed within an existing supervision structure, were consistent with the goals of supervision, and did not add additional meetings for providers. In recently established, school district-operated programs, the roles of many school staff often changed dramatically and rapidly over the course of the study resulting in the requirement for them to provide counseling services in addition to their usual responsibilities. In these contexts, adjusting to new roles, changes in staffing, and establishing systems took immediate priority, impeding readiness to learn and deliver a new EBI. In the context of contracted agencies, staff turnover led to a need for additional training and consultation for new staff, but also facilitated wider EBI dissemination as trained staff took their knowledge to new programs. Although provision of logistic and leader support of training and EBI use facilitated implementation in both MH and ED settings (see above), this was highlighted more in MH (e.g., space and scheduling support to recruitment and training activities).

Inner Context Factors: Providers (Teachers/Therapists)

Provider factors including provider motivation, attitudes, and general clinical skills were relevant and particularly highlighted in trainer focus groups. Although most factors were common or shared across service settings, they had a differential impact on the implementation process in MH and ED settings.

Shared Influences

Previous training and experience (particularly in behavior management) was considered important for both MH and ED providers. For MH providers with previous training in behavioral or skill-focused interventions, such that the language and terminology were more familiar to them, were perceived by trainers to have an easier time learning the new strategies as the new EBI was closer to their existing practices. MH providers without that training were perceived to have more challenges with the basic elements of the intervention (for example, the new behavioral terms) which initially served as a barrier. Similarly, for the ED setting, teachers perceived to demonstrate foundational teaching skills such as classroom routines, student behavior management systems were perceived to be more readily able to deliver a new EBI into their program than teachers who did not have these indicators of classroom quality. Likewise, experience was noted by MH trainers to both positively and negatively impact the training process. MH and ED Trainers reported that very new providers could be more focused on learning and managing time and the demands of a new job than delivering a new intervention, whereas those with many years of experience often made more adaptations to the EBI protocol based on their own experience.

During the focus groups, trainers across both MH and ED discussed their observations of providers having an optimal level of provider anxiety during the implementation phase such that the provider was motivated to learn strategies to appropriately serve children with ASD, but not so anxious that it interfered with receiving and accepting feedback or being video recorded. Trainers also commented on the reduction of initial anxiety after providers used the EBI strategies and found them to be effective. Interestingly, both MH and ED trainers emphasized providers’ general motivation and conscientiousness as impacting engagement in training and intervention delivery.

Unique Influences

Although provider motivation was considered a key factor in both settings, it was considered more salient in MH. Specifically, there were repeated comments reflecting the perception that MH providers felt an urgency for ASD training due to not feeling adequately trained to serve children with ASD. If providers had a child with ASD on their caseload during therapist/client recruitment, they were often highly motivated to participate in the training. Teachers, in contrast, were perceived to feel they had more training around ASD in general as they consistently taught children with ASD and displayed less urgency for training.

Perceptions of the Research, Intervention, and Developer

Shared Influences

Provider workload and perceived burden of participation in the research was perceived to affect enrollment in both contexts (for example, teachers were observed to consider identification and recruitment of child participants as a challenge in the context of other demands). These factors continued to influence providers during implementation, as demonstrated by level of engagement in training (attendance). MH providers who did not regularly attend ongoing consultations with clinical trainers often cited workload and competing demands as barriers to attendance. ED trainers also perceived the burden of participation and workload also noted as a key factor, and indicated teacher attendance in training and coaching activities as negatively affected by their workload and contributing to discontinuing participation in the study during implementation.

Client factors, including the intervention fit with client needs and populations, emerged as a strong determinant of implementation in both contexts. The fit of AIM HI to address a significant unmet need regarding training in specialized ASD interventions was a significant facilitator during both the preparation and sustainment phases. For ED services, the fact that CPRT was teacher-developed intervention helped to better meet range of students seen in in these settings; however, teacher perceptions regarding limited fit with client functioning or age (e.g., client too low or high functioning, client too young) served as a barrier.

Intervention characteristics, specifically related to provider perceptions regarding flexibility, materials and adaptive training structure were seen as beneficial in both studies. For example, therapists reported that the AIM HI strategies were useful for parents, therapists and teachers due to the flexibility in the protocol and found this helpful for service coordination. Teachers using CPRT indicated that having flexible data collection methods that could be integrated into methods already used by the school program was useful. Broader system and program leader perceptions regarding flexibility regarding the intervention model were key facilitators in MH services as the intervention could flexibly fit within the funding structure and could be utilized by the providers from a range of disciplines and backgrounds.

Unique Influences

For MH services, the relationships between the intervention developers and their research teams and the participants were considered to facilitate involvement. Building on existing relationships with MH system and agency leaders was perceived to facilitate adoption and the research teams’ willingness to be flexible and individualize study activities (agency enrollment, provider/child recruitment, provider training) for different contexts and participant types was perceived as critical in success of these activities.

Discussion

This study highlights the potential for hybrid effectiveness/implementation studies to advance the field’s understanding of implementation processes of EBI in routine care settings (Curran et al. 2012) and illustrates the application of a relevant implementation framework to guide study conceptualization, measurement, and interpretation of findings. This study involved coordinating prospective qualitative data collection on project implementation teams (including intervention developers’ (in this case also the study PIs) and research trainer) perspectives on the implementation process to inform future training and implementation processes within the context of two ASD interventions and service delivery contexts. We applied the EPIS framework to characterize the shared and unique outer and inner context factors influencing implementation during community effectiveness trials. Results indicate that many of the themes, particularly the most salient themes, were common across MH and ED settings (leader engagement, provider attitudes and experience). The unique themes were additional influences highlighted in MH programs only. These primarily involved structural considerations for each setting (e.g. maturity of MH setting related to readiness for new EBI) and some specific considerations related to the intervention characteristics. While the unique factors may influence specific implementation planning decisions for MH settings, the common factors highlight promising generalizable targets of implementation strategies for ASD EBIs that can be leveraged within growing, large-scale translation efforts in usual care.

Specifically, our findings suggest that collaborative adaptation of EBIs can lead to EBIs that are viewed as more flexible and have a good fit with the client and community context. Collaborations have recently been incorporated into EPIS as “bridging factors” that were identified in a systematic review of 48 studies that employed the EPIS framework (Moulin et al. 2018). One of the ways in which collaborations manifest can be through collaborations of intervention developers, with researchers and community stakeholders. This, combined with another new EPIS domain of “innovation factors,” can be parlayed through collaboration to appropriately adapt interventions for specific contexts. These intervention factors were seen as facilitators of implementation and sustainment of the interventions. A growing body of literature suggests that research-community partnerships are a promising method to increase effective implementation and sustainment of EBI in community ASD services (Brookman-Frazee et al. 2016; Drahota et al. 2016). It is likely that the involvement of community stakeholders at all phases of intervention development / adaptation will lead to increased use of the interventions with fidelity. Future research can help determine the specific mechanisms of action by which partnerships facilitate the implementation process.

However, flexible and feasible interventions are not always enough. In this study, outer context and program/district factors (e.g., type; leadership) clearly affected implementation. These data support the importance of considering organizational readiness for adopting an EBI. Implementing a new EBI in the context of a large system-wide policy change can prove challenging and may require taking additional pre-implementation steps to build a structure for training and supervision as well as time for defining staff and leader roles in the implementation process. Adding a new innovation to a system that has a strong method for training and oversight appears to increase implementation as compared to introducing an EBI in a system that has no experience training or rolling out new skills to the team. A better understanding of the underlying structure needed in different systems of care to support EBI prior to training and EBI implementation may increase EBI use and sustainment and be more cost effective in the long run. Consideration the relationship between specific support structures (e.g., implementation leadership; systems of supervision; time in schedule for EBI training and practice) and intervention fidelity may provide information about the key aspects of support needed for success.

Another salient influence across service contexts and phases was leader engagement in the implementation process. In the Preparation phase, system leader motivation and involvement was considered critical by study investigators in facilitating recruitment of large numbers of programs and districts in the outer context. In the inner context, program leader facilitation was critical in implementation planning and roll-out. These findings are consistent with the EPIS framework where both outer and inner context leadership are important determinants of downstream buy-in and implementation climate (Aarons et al. 2014). In addition, our findings are consistent with research that highlights the importance of leadership in successful implementation of innovative practices (Bass and Avolio 1990; Edmondson 2003; Klein et al. 2001; Powell et al. 2012; Stogdill 1974). When leaders provide clear guidance during implementation, facilitate support among co-workers and from administration for effective implementation, trainees report an increased sense of competence and satisfaction (Green et al. 2014). Consistent with findings from recent mixed-methods research, it is also clear that leadership across levels is critical for implementation and sustainment (Aarons et al. 2016). For example, at the system level it is important for leaders to clearly establish a project’s mission and vision, engage in early and continued planning for EBI sustainment, develop and follow a realistic implementation plan, and identify multiple strategies for sustainment (Mancini and Marek 2004). At the EPIS inner context organizational level it is important to have “transformational” leaders who can inspire and motivate staff while taking into account their individual needs and motivations. Leadership that is passive and/or avoidant, however, can threaten implementation and sustainment (Aarons et al. 2016).

Another salient EPIS inner context factor was provider attitudes and experience. Our findings regarding provider attitudes are consistent with mixed methods findings from our complementary data on MH provider perspectives on implementation (Dyson et al. 2017). Specifically, we found that although successful and unsuccessful providers (i.e. those who successfully complete vs. do not complete the 6 month AIM HI training period) both face challenges (e.g., time; program structural challenges) during the implementation process, they differ in their perceptions of the impact of these challenges on implementation Successful providers were more likely to perceive challenges to be difficult, but ultimately manageable, aspects of training, while unsuccessful providers perceived challenges to be insurmountable. These perceptual differences were associated with changes to therapist behavior (e.g., reduced attendance at training consultations) and ultimately impacted training outcomes. Although many factors influenced EBI training outcomes, we found it was the therapists’ motivation for training, attitudes towards the training experience, and their perceptions of their own ability to respond and adapt to challenges faced during training and intervention delivery that had the greatest impact on training outcomes.

Not surprisingly, providers who had some familiarity with behavioral concepts were perceived to have an easier time learning the EBIs. Similar to organizational readiness, providers needed to have a baseline level of knowledge and, for teachers, classroom structure, upon which to apply the EBI. In addition to highlighting the need for increased exposure to EBI knowledge and implementation during professional training for providers, this suggests that pre-service training programs have a role to play in preparing providers with foundational knowledge and skill such that they are prepared to learn new strategies as evidence of efficacy becomes available. Similarly, intervention developers might have greater success if new EBIs are situated in the context of current care and use language familiar to the providers. Again, this suggests an important role for research-community collaboration.

These findings should be considered in the context of the lens in which they were obtained. Namely, this study examines implementation processes from the researcher/intervention developer perspective. The perspectives provided are limited to those of the research team and intervention trainers. These finding build on prior qualitative work examining caregiver, provider, leader perspectives, however, these viewpoints are not represented in the current analyses which may mean we are missing specific facilitators and barriers that may also affect the implementation process of their perspectives. These data provide complementary data to support next steps in implementation across these systems of care and will facilitate scale up of the interventions.

This study has a number of important strengths and is innovative in the coordination of data collection in independent, concurrent effectiveness studies incorporating the perspectives of multiple research stakeholders, including PIs (intervention developers), study coordinators, and trainers.

Conclusions and Future Directions

EBI intervention implementation and sustainment is a complex process that involves interactions and relationships between intervention developers, and community stakeholders including system, organizations, and service providers. The use of the EPIS framework was applied to identify and organize both outer and inner context factors that may impact implementation across the phases of the implementation process. AIM HI and CPRT research share common methods for developing, adapting, and testing interventions in varied community service settings (Wood et al. 2015) and reports similar themes in implementation processes and outcomes, providing a unique opportunity for a cross-service setting comparison of innovative implementation interventions. In particularly, themes from the independent effectiveness studies indicated that provider attitudes and implementation leadership are promising targets of implementation interventions. As such, our groups are now conducting two, coordinated studies testing the effectiveness of the “Translating Evidence-based Interventions (EBI) for ASD: Multi-Level Implementation Strategy” (TEAMS) model (R01MH111950 and R01MH111981) (Brookman-Frazee and Stahmer 2018). TEAMS targets implementation leadership, organizational climate, and provider attitudes and motivation in order to improve two key implementation outcomes—provider training completion and ASD EBI fidelity, and subsequent child outcomes. The TEAMS Leadership Institute applies the LOCI (“Leadership and Organizational Change for Implementation”; Aarons et al. 2015) strategies, and the TEAMS Individualized Provider Strategy for training (TIPS) applies MI (Motivational Interviewing) strategies to facilitate individual provider and organizational behavior change. These studies are using a randomized implementation/effectiveness Hybrid Type 3, trial to test TEAMS model with the AIM HI in publicly-funded mental health services and CPRT intervention in education settings. A dismantling design is used to understand the effectiveness of TEAMS and the mechanisms of change across settings and participants. Implementation outcomes (Proctor et al. 2009, 2011) including provider training completion, fidelity and child behavior change will be examined. This implementation intervention has the potential to increase quality of care for ASD in publicly-funded settings by improving effectiveness of EBI implementation, however, the process and modules will be generalizable to multiple service systems, providers, and interventions, providing broad impact in community services.