Introduction

One in four US children experiences a mental health disorder with severe impairment and/or distress during their childhood (Costello and Egger 2005; Kessler et al. 2011; Merikangas et al. 2010). Ensuring that these children have access to the highest quality mental health care is a top public health priority. Efficacious child and adolescent treatments exist for a wide range of mental health disorders (Chorpita et al. 2011). However, these treatments are rarely delivered with satisfactory fidelity in community-based mental health service settings (Drake et al. 2003; Garland et al. 2010; Kazak et al. 2010; McHugo et al. 2007; Raghavan et al. 2010). This is not just a quality shortfall, but a quality chasm (McCabe 2004), as an Institute of Medicine report concluded in a review of health care and mental health studies, stating that “there are large gaps between the care people should receive and the care they do receive” (Institute of Medicine; IOM 2001, p. 236). Statewide and national dissemination and implementation initiatives are underway to narrow these gaps (Ebert et al. 2011; Karlin et al. 2010; Saunders 2009). However, the problem of provider fidelity continues to present major challenges that are critical to address if care is to be improved.

Fidelity to evidence-based treatments (EBTs) is variable, even among well-trained providers (Drake et al. 2003; McHugo et al. 2007; McCabe 2004; IOM 2001; Saunders 2009). This is a major public health concern because high EBT fidelity is associated with better patient outcomes (Bond et al. 2011; McHugo et al. 1999; Schoenwald et al. 2004). Achieving a greater understanding of the challenges experienced by providers when implementing empirically supported mental health treatments, as well as the potential value of innovative solutions, is therefore a top priority. To this end, we conducted a qualitative study with 19 trainers who successfully completed a training program in Trauma-Focused Cognitive Behavioral Therapy (TF-CBT; Cohen et al. 2006), led by the treatment developers. TF-CBT is an evidence-based treatment that addresses symptoms of posttraumatic stress, depression and disruptive behavior among children exposed to traumatic events. TF-CBT is an ideal treatment model for a qualitative study of this nature because it is among the most well-established treatments for children in the mental health field, and because it uniquely addresses multiple symptom domains (e.g., posttraumatic stress, depressed mood, disruptive behavior). In this study, we assessed approved national trainers’ perspectives about challenges that providers experience when implementing TF-CBT, and explored their perceptions about the potential value of innovative, technology-based solutions to enhance provider fidelity and improve quality of care.

Trauma-Focused Cognitive Behavioral Therapy is a short term treatment model, comprised of eight specific treatment components that comprise the acronym “PPRACTICE” and includes Psychoeducation and Parenting skills; Relaxation; Affective expression and modulation; Cognitive coping; development and processing of a Trauma narrative; In vivo exposure; Conjoint sessions where the child shares their Trauma Narrative (TN) with a supportive caregiver; and strategies to Enhance future safety and development. TF-CBT has been evaluated in numerous randomized controlled trials and is among the most well-established and widely disseminated mental health treatments for children (Cary and McMillen 2012; Cohen et al. 2004, 2011; Deblinger et al. 2011, 2006).

Method

Participants

Individual thematic telephone interviews were conducted with a sample of 19 mental health professionals who successfully completed a 15 month TF-CBT Train-the-Trainer program led by the TF-CBT developers (Drs. Cohen, Deblinger, and Mannarino). Because of their extensive training activities, all trainers worked closely with front-line providers and had first-hand knowledge of challenges to delivery of TF-CBT with fidelity. They were therefore ideally positioned to provide valuable qualitative data about common barriers and challenges faced by community-based providers, as well as to provide insight into the potential value of technology-based resources designed to improve quality of care. Collectively, the 19 professional trainers in our sample had been TF-CBT trainers for at least 3 years (M = 5.8, SD = 1.8), had at least 8 years of experience treating children (M = 17.1, SD = 5.2), and trained over 5,000 providers during the past year alone (M = 293.7, SD = 190.7). Most of the trainers (89 %) engaged in multi-component training activities that included in-person workshops and monthly or bi-monthly consultation calls with participating providers. Trainers were from all regions of the U.S.; most were women (83 %) and non-Hispanic White (81 %). The sample was evenly distributed between master’s- and doctoral-level degrees; all were licensed mental health professionals.

Interview

The telephone interview consisted of two major sections: a brief section on interviewee demographics and experience (approximately 5 min), followed by a thematic interview assessing trainers’ perceptions about challenges experienced by providers when delivering TF-CBT (approximately 30 min). The thematic interview was semi-structured and addressed several issues relevant to implementation of TF-CBT with fidelity. First, trainers were asked specifically about TF-CBT components that they believed were most challenging for providers to deliver with a high degree of fidelity and competence. Follow up probes asked the trainers for their perceptions about why these specific components were difficult for providers. Second, trainers were asked which TF-CBT components they thought were the most difficult to engage children or caregivers. Follow-up probes inquired about the trainers’ perceptions on why these components were challenging for engagement. After these issues were addressed, trainers were asked an additional series of questions about what they perceived as the potential values and roadblocks associated with using technology-based resources to enhance provider fidelity and competence and patient engagement. This was described to them as follows:

One of the goals of our project is to develop technology-based tools to help providers deliver a high quality of care and keep families actively engaged in TF-CBT components, particularly those that are known to be the most challenging to deliver. The tools would be available on a tablet such as an iPad, and would include things such as interactive educational games, a trauma narrative writing tool, and video demonstrations to show caregivers specific behavior management skills.

Follow-up probes inquired about what features or topics should be included or excluded to ensure that these resources were engaging to children and their caregivers while also being useful to providers. Additional probes were used to generate ideas to ameliorate potential roadblocks to use of technology-based resources and to assess how their use might affect the therapeutic relationship. A copy of the full thematic interview is included in the Appendix.

Procedure

After IRB approval was obtained, we sent a single e-mail invitation to the full group of approved TF-CBT national trainers (n = 51). We aimed to conduct between 15 and 20 interviews to achieve saturation (Morse 2000). Participants were not offered compensation. Thirty trainers (59 %) responded to our e-mail invitation and agreed to be contacted for interview. We were successful in scheduling interviews with 25 of these trainers, resulting in 21 completed interviews. Two interviews were not able to be coded for analysis due to audio-recorder malfunction, resulting in a total of 19 coded interviews. A postdoctoral fellow with training in, and experience delivering, the TF-CBT model administered the interviews. Interviews were 20–55 min (M = 33) and were audio-recorded. Interviews were transcribed, coded, and checked for accuracy.

Results

Data Analyses

Given the limited research on trainers’ impressions of provider fidelity and competence, the qualitative approach chosen for this analysis is the constructivist grounded theory comparative method to code incident-by-incident (Charmaz 2006). This approach was well suited for these data, as it takes the context of the question into account, acknowledges coders’ prior knowledge of research related to fidelity and competence and the influence of this experience on the coding processes, and provides guidelines for building conceptual frameworks between coded constructs. A doctoral level psychologist conducted primary line-by-line thematic coding using NVivo-9 (2010) to mark identified thick descriptors. Secondary and focused coding were then conducted to impose superordinate thematic codes and hierarchical structure through the methods of constant comparison and analytic induction (Glaser and Strauss 1967). This process ensures both consistency in the coding and theme labeling within categories and divergence between coding categories. These results yielded 326 thematic codes. Consistent with Miles and Huberman’s (1994) guidelines for similar sample sizes, only codes that held for at least three transcripts were retained, resulting in 249 themes. Themes were then organized into 17 superordinate thematic codes based on their presence, co-occurrence, and non-overlap in the transcripts.

The superordinate thematic codes (presented in Tables 1, 2, 3, 4) fell into 4 larger domains: Fidelity, Competence, Engagement Barriers, and Opinions about the Technology-based Toolkit. During this stage of coding, Fidelity (Domain 1) was defined as provider problems in use of specific TF-CBT components; Competence (Domain 2) was defined as provider problems in knowing how to deliver the TF-CBT components well; Engagement barriers (Domain 3) were defined as problems getting patients to start, complete, or participate consistently in treatment and were divided further into predominantly caregiver versus child barriers. For each of these 3 domains, specific TF-CBT model component superordinate themes were coded based on which of the TF-CBT model components (e.g., Psychoeducation, Relaxation) were indicated. Opinions about the Technology-based toolkit (Domain 4), defined as provider comments about the proposed toolkit, was further divided into anticipated barriers or problems with use of the toolkit, general benefits of the toolkit, and/or problems the toolkit may solve.

Table 1 Domain 1: themes related to provider fidelity
Full size table
Table 2 Domain 2: themes related to provider competence
Full size table
Table 3 Domain 3: themes related to child and caregiver engagement
Full size table
Table 4 Domain 4: themes related to opinions about e-resources for TF-CBT
Full size table

Inter-Rater Reliability

Approximately 20 % of interviews were randomly selected and coded by a second, doctoral level, coder to assess overall inter-rater reliability. As recommended by Bakeman and Gottman (1997), discrepancies in coding between the primary and secondary coder were resolved through discussion until consensus was reached. These assessments were compared using NVivo-9 to calculate transcript level kappa coefficients and then computing average kappa’s for each theme. Kappas above 0.60 are rated as reliable (Pellegrini 2004) and indicate consistency in coding occurrence, commission, and omission, adjusted for chance. Of the 249 themes that resulted from the secondary coding, 219 had good-to-excellent levels of inter-rater reliability and were retained (κ range 0.61–1.00, κ M = 0.95). Thus, only 30 themes were unreliable; these were not included in discussion. Kappa values for the 17 retained superordinate thematic codes ranged from 0.61 to 1.0 (M = 0.88), also indicating good-to-high levels of inter-rater agreement. The resulting reliably coded themes, number of trainers that addressed each theme, and the number of references to the theme across the sample are presented in Tables 1 through 4.

Verification

The resulting data interpretations also were reviewed for validity through member checks (Kvale 1996; Manning 1997). In this case, several TF-CBT trainers, the interviewer who conducted the thematic interviews and the TF-CBT developers reviewed the identified themes and gave feedback as to their validity. Our data interpretations were reported to be generally consistent with their impressions.

Domain 1: Provider Fidelity

As seen in Table 1, most of the trainers believed that there were some problems with whether or not providers implemented specific TF-CBT components, which we categorized as ‘fidelity’ to the model. These concerns fell into several TF-CBT components specifically related to trauma narrative development (13 trainers), cognitive coping (12 trainers), psychoeducation (6 trainers), parent behavior management (5 trainers), and enhancing safety (3 trainers). In identifying enhancing safety as a component that poses a challenge to fidelity, one trainer commented, “Well, it is not a difficult component, but it is that providers forget it like you drop it off the end of the model and, you know it comes right after the narrative session which a lot of providers have a really big build up to.” One explanation for not implementing the trauma narrative was related to provider avoidance. As one trainer noted, “Yeah, I mean the trauma narrative is difficult because I think there’s a lot of avoidance on behalf of the clinicians. They’re concerned that they’re being mean. They’re concerned that the child’s not ready….”

Domain 2: Provider Competence

Most of the trainers believed there were significant problems in providers’ understanding of and abilities to deliver EBTs more generally, and TF-CBT more specifically. We categorized these responses as problems in ‘competence.’ As one trainer explained, “I think there are a lot of clinicians who have a very nondirective approach to therapy and I think they really struggle with aspects of TF-CBT because of that.” Another trainer noted, “I think that is what gets in the way of cognitive processing, too. They do not have enough training themselves in how to do basic CBT behavioral analyses and [cognitive] processing.” Several TF-CBT components identified by the trainers as containing skills that providers either had difficulty doing well or did not fully understand are common to a variety of cognitive-behavioral EBTs. For example, trainers had concerns about providers’ skills in using cognitive therapy generally (14 trainers), exposure sessions (12 trainers), challenging negative thoughts in cognitive processing (10), addressing (8 trainers) and appropriately responding to avoidance (8 trainers), and flexibly adapting the model (7 trainers) to the specific patient.

Regarding TF-CBT specifically, trainers identified development and processing of the trauma narrative (10 trainers) and psychoeducation (5 trainers) as components that posed challenges to provider competence. Trainers’ concerns about psychoeducation were related to providers talking at the child rather than with them about the trauma (5 trainers). For the trauma narrative, trainers were concerned about providers’ abilities to: determine the necessary level of detail (6 trainers), respond appropriately to child emotions (7 trainers), deal with their own fear and avoidance (4 trainers), direct the cognitive processing of the narrative (6 trainers), and facilitate sharing of the narrative with the caregiver (5 trainers). A number of trainers noted that challenging negative thoughts during cognitive processing of the narrative was particularly difficult (10 trainers). One trainer stated,

I think that these skill sets are probably the least trained skill sets in clinical programs and so I think that the therapist themselves just have the least comfort in applying these skills…You know it’s one thing to kind of try to do cognitive processing and it’s another thing I think to be skilled and adept at it. I think that they just haven’t had good solid training in that.

Similarly, another trainer stated,

In terms of the cognitive processing component, I think that’s a skill set that’s the most new in terms of the skill set for most trainees that I’ve been working with…it’s harder for them to sort of learn and wrap their brain around what the goal is and how it’s done most successfully.

Domain 3: Client Engagement

Although many trainers noted that child engagement was generally not a concern (6 trainers), others suggested that avoidance posed challenges with some of the model components (Table 3). Of particular note, trainers expressed their belief that trauma avoidance is a significant issue in child engagement (7 trainers) and that avoidance of trauma cues at the onset results in children disengaging before treatment even really begins (4 trainers). One trainer explained, “There are very reserved children who are mostly avoidant. That can be difficult at all the phases of the model because the child is scared or resistant or not engaged in the therapy process overall.”

Whereas avoidance also was an issue for caregiver engagement (6 trainers), other issues emerged. Specifically, several trainers noted that caregivers’ own mental health problems can negatively affect engagement in treatment. One trainer noted that caregivers’ mental health problems require that providers put in considerable extra work, stating that.

…a lot of times the parents are so low functioning that they are just dealing with their own trauma. Dealing with case management and handling these things up front- if you don’t handle them, then it can become a big barrier down the line to doing the rest of treatment; they actually at some point undermine what the therapist is trying to do with psycho education for the child.

Trainers also highlighted caregivers’ inabilities to buy into treatment as a major barrier to engagement (7 trainers). One trainer noted that many caregivers believe treatment should only involve the child (3 trainers), making it difficult to convey that “helping them (the caregiver) understand that in this treatment there is a parallel parent component and that the degree to which you can help your kid get better is depending on their participation.”

Domain 4: Opinions About the Proposed e-TFCBT Toolkit

Finally, we inquired about trainers’ beliefs regarding the utility of an e-based toolkit (e.g., IPad/tablet) to facilitate TF-CBT implementation and whether they perceived any barriers to its use (see Table 4). Overall, trainers shared positive opinions about the idea of a tablet (13 trainers), stating that it had the potential to improve the child-provider relationship (11 trainers), as well as positively impact fidelity (9 trainers), competence (9 trainers), and engagement (9 trainers). As one trainer explained,

…I think the less the clinicians have to worry about the mechanics of what they are doing, and find adequate resources to do so, I think that allows them to relax a little bit more and pay attention to the engagement and potential challenges or threats to engagement.

Another trainer noted,

I think kids would feel very engaged. The child’s generation connects through technology, right? I mean, they actually connect much less face to face. So, I think that it’s embracing that generation. I think it makes therapy possibly more accessible to a kid who is used to connecting with friends through texting and Facebook.

Beyond provider buy-in, trainers also believe this tool can enhance children’s beliefs in the effectiveness of TF-CBT (4 trainers). Trainers also indicated that this tool might have tremendous utility for treatment delivered in non-traditional, community-based settings (3 trainers). As one trainer highlighted,

…A lot of clinicians (who) go to homes or, you know do outpatient care in the community at homes or schools. Then I could see it being a huge benefit there if they could take it with them so that they do not have to bring their big filing box with all of their different themes and activities….

In contrast, many trainers noted that an e-toolkit could be a barrier to therapy. For example, one trainer stated,

I think it is marvelous that we have lots of tools and resources at our disposal to help playfully and interestingly engage our clients, and to see how people are interested in things like electronic media and apps for phones and things that are attractive and novel…but it can actually end up serving the role of reinforcing avoidance … my biggest concern that these things don’t get to be seen as the end in themselves and they are simply a means to an end.

Additionally several providers expressed doubt regarding the usefulness of a tablet-based toolkit for providers, many of whom they believe are uncomfortable with technology (8 trainers). One trainer stated, “I think there’s always going to be those therapists who are less comfortable with online technology but I see that becoming a smaller and smaller problem.” Several trainers noted that if personal information could be entered into the application, this would lead to privacy problems (5 trainers), would become problematic for the privacy of the trauma narrative (4 trainers), and that they would be unwilling to use the application (6 trainers).

Some trainers also were concerned that providers may rely too heavily on a tablet-based toolkit when delivering TF-CBT instead of using it as a supplemental tool (5 trainers) and that this might negatively affect the provider-patient relationship. Specifically, trainers worried that if rapport and competency were already low, a toolkit could further hurt the provider-patient relationship (4 trainers) and that some providers might use the tablet computer/iPad as a ‘babysitter’ for child patients (5 providers). Similarly, many trainers were concerned that providers would think that using the toolkit alone would be sufficient to call the treatment ‘TF-CBT,’ leading to providers deviating from the protocol and having poor fidelity to the model (7 trainers). As a way to address this problem, one trainer highlighted the importance of developing a toolkit so that “it somehow communicates that this is an enhancer and is not a replacer.”

Finally, trainers noted that providers may lack access to the equipment necessary to use an e-toolkit, such as an iPad/tablet (7 trainers), desktop computer (6 trainers), or internet access (6 trainers). One trainer stated, “Okay, well the biggest challenge I know even for our agency would be that we don’t have computers in our therapy room.” Trainers expressed concern about the cost of the devices and the required internet, stating that regular therapists could not afford these devices (6 trainers), that patients would not have access to them or to the internet to use them in their homes (5 trainers), and that agencies would be unwilling to provide them due to their high cost (5 trainers).

Discussion

The primary aims of this study were to identify the most common challenges experienced by providers in the implementation of a trauma-focused evidence-based mental health treatment. TF-CBT (Cohen et al. 2006) was selected as an exemplar because of its extensive empirical support as well as our recognition that it shares common core components with other evidence-based, cognitive-behavioral mental health interventions for youth. In the present study, thematic interviews were conducted with a sample of key stakeholders, approved TF-CBT national trainers. These trainers have extensive involvement in training front-line mental health providers as well as significant direct experience in delivery of the model, which positioned them well to provide valuable data regarding common challenges experienced by providers in treatment delivery.

Trainer Perspectives on Challenges to Provider Fidelity, Competence and Child/Caregiver Engagement

Consistent with previous research (e.g., Allen and Johnson 2012), many of the trainers reported that gradual exposure and cognitive coping presented significant challenges to treatment fidelity and provider competence. Explanations included provider discomfort in directive treatment approaches, as well as limited skill in delivering treatment and engaging patients in these components. Despite extensive data indicating the effectiveness of these treatment components (Bisson 2007; Carr 2004; Deblinger et al. 2011; Silverman et al. 2008b, a; Wethington et al. 2008), many providers express discomfort and fear that these components will cause harm and undue distress. Concerns that such discomfort and fear can affect the fidelity and quality of care were reinforced by findings of the current study. Innovative solutions are clearly needed to identify new techniques that can assist providers in overcoming these barriers—for example, via specialized training and practice. Research exploring new directions in this area may have far-reaching implications.

A second focus of the interviews was to examine challenges to engagement. Although many trainers did not believe child engagement was problematic, those that did frequently identified patient avoidance as the specific barrier to engagement with both children and adults. Thus, as a means of enhancing engagement and reducing the likelihood that avoidance can negatively affect treatment, it is important to address these issues directly at the outset and throughout the treatment process and to provide a strong rationale for all of the treatment components (Cohen et al. 2010, 2006). It is also important to ensure that providers are appropriately trained to recognize and creatively address subtle forms of avoidance that may be driving disengagement. Due to the multi-component, complex structures of many child mental health treatments, these types of critical skills are often underemphasized in training, and deficits in these skills often go unrecognized in practice.

Trainers also believed caregivers’ own avoidance interfered with their abilities to engage in treatment. Other specific challenges to caregiver engagement included caregivers’ insufficient beliefs in the treatment process/rationale, their own symptoms or mental health problems and/or their beliefs that their child should be the sole focus of therapy. Whereas research has emphasized the importance of caregiver involvement to facilitate positive clinical outcomes (Dowell and Ogles 2010; Deblinger et al. 2006, 1996), data from the current study indicate that engaging the caregiver can be especially challenging. Thus, it is important that providers develop the skills to recognize situations where a caregiver’s own mental health symptoms interfere with engagement in treatment, and to make referrals for individual treatment when indicated. Providers need to highlight the integral role of the caregiver in the therapeutic process from the outset of treatment and explicitly and repeatedly provide the rationale for the treatment approach. These issues are relevant to many therapeutic interventions for children because of the integral role of the caregiver in the child’s recovery (Dowell and Ogles 2010; McKay et al. 2001). Finally, it is important to highlight that empirically supported strategies have been identified to address logistical and perceptual barriers to initial family engagement (McKay and Bannon 2004; McKay et al. 2005) and to enhance active participation throughout all phases of the treatment process (Tuerk et al. 2012). These should be incorporated as key components of child and family-based treatments.

Trainer Perspectives on the Potential Value of Technology-Based Resources

Many of the trainers noted that a tablet-based toolkit could assist providers in overcoming some of these identified challenges to the child-provider relationship, treatment fidelity, provider competence, and child and caregiver engagement. For example, a notable threat to provider fidelity is lack of engagement with patients at various phases of treatment. The use of innovative, technology-based learning activities in session offers a possible way to keep patients engaged and to reduce potential boredom and disinterest in the treatment exercises, particularly when working with children and adolescents (Matthews et al. 2008). Use of an iPad, computer, or other mobile or web-based device has the potential to increase engagement, particularly given children’s growing familiarity and comfort in an increasingly technology-driven world (Rainie 2009). Youth often have ready and instant access to technology in all phases of their lives, making it likely that this would enhance their interest, investment, and engagement in the therapy process.

Regarding provider fidelity, inclusion of material within a tablet-based resource that directly maps on to the specific model components may help keep providers on track, facilitate progress through the treatment model, provide reminders of treatment goals, and generate creative activities and ways to supplement the key principles/techniques being taught in therapy. These resources also have the potential to reduce therapist drift by providing tangible reminders of key treatment components.

Whereas many trainers positively endorsed the use of technology, there were also reservations. Predominantly these related to concerns that agency-based or private-practice providers have poor access to the internet and/or computers (tablet or otherwise) either in their office or while engaged in community-based outreach services, limiting their ability to use this type of resource. Trainers also expressed apprehension about cost as well as general openness and skill in using technology, making it possible that these resources would only be accessible to a certain segment of the population and thereby limit widespread dissemination and reach. Research that develops and evaluates these types of technology-based resources should prioritize cost-benefit analyses to ensure that sufficient data are available to agencies to assist them in estimating the value of such investments to their patient population and in terms of their ability to offer high-quality care.

Questions regarding confidentiality, web security, and general privacy also were raised as potential drawbacks to the use of an e-based toolkit. Thus, any e-based toolkit must include significant safeguards to protect privacy and confidentiality. And finally, trainers indicated that fidelity may be compromised if providers mistakenly believe that sole reliance on a toolkit is equated with successful completion of a complex, multi-component treatment model, such as TF-CBT. This highlights the importance of educating providers on the judicious use of a technology-based resource as a way to enhance and supplement, rather than detract from, the therapy process.

Limitations

This study had several limitations. First, thematic interviews were conducted with a small sample of professionals engaged in a specific evidence-based treatment protocol. Whereas the core TF-CBT components (psychoeducation, parenting skills, relaxation strategies, cognitive coping) are common to many other EBTs used in youth mental health, this is still a specialized treatment approach. It is possible that these findings may have relevance to other child-oriented CBTs, but it is premature to draw this conclusion without additional research.

Second, these data were provided by trainers regarding their perceptions of provider behavior. In most cases, trainers were relying on information drawn from provider self-report and questions raised during consultation calls or workshops, discussions with other trainers, or behavioral rehearsal activities during training workshops. It is likely that a minority of the trainers had first-hand knowledge of provider behavior via direct observation of treatment sessions with actual patients. However, this was not directly assessed in the current study, and the extent to which trainers’ perceptions were based on direct observation of provider behavior is unclear. Third, few inferences can be drawn about trainers’ perceptions of providers’ strengths in delivery of TF-CBT because the emphasis of the interview was on problems or challenges to implementation.

A final limitation concerns the definitions of fidelity and competence used in the current study. As noted in the literature (Fairburn and Cooper 2011; Perepletchikova et al. 2007; Schoenwald et al. 2011), there are considerable challenges to the reliable and valid measurement of these constructs, and, in the present study, no specific guidelines or definitions were provided to interviewees to insure that they were differentiating between these constructs. However, inter-rater reliabilities were in the good-to-excellent range (κ range 0.61–1.00, κ M = 0.95) indicating that interviewees and coders were able to make these distinctions. Further, there appeared to be differences regarding respondents’ opinions on the types of challenges faced by providers in choosing to use a model component (i.e., fidelity) versus delivering the model with competence. It is likely that greater focus on therapist skill results in greater model adherence, and this continues to be an important focus for future research. But, perhaps the most important issue is to determine the ‘tipping point,’ namely at what point is a treatment delivered ‘well enough’ to achieve a desired clinical outcome? Fairburn and Cooper (2011) refer to this as ‘therapy quality’ and highlight that this may be more critical than definitional distinctions, particularly in clinical practice settings. We concur that this is an important and fruitful area for future research.

Conclusions

These data offer some important insights and implications for training in EBTs, provider fidelity and competence, and patient engagement. Trainers were generally consistent in their beliefs that certain CBT components pose more challenges to providers than others. Thus, it may be important for trainings to provide more in-depth focus on these key components—for example, specific content and opportunities for behavioral rehearsal in gradual exposure and cognitive processing components appear to be integral to helping providers achieve and maintain fidelity to this type of treatment approach. Further, it appears that trainers should not overlook the importance of ensuring that providers have basic CBT skills, rather than making a priori assumptions about the skill level of training participants. This speaks to the importance of pre-training assessments as well as engagement and discussion with agency leaders, supervisors, and the providers themselves to determine the level of provider skill prior to training and to incorporate basic therapy skill training if this is indicated.

Third, recent technological advances offer an opportunity to improve quality of care significantly by supporting the effective delivery of best-practice interventions. Such resources should be easy to use and should support the therapeutic alliance to allow efficient and effective integration into everyday practice. A technology-aided approach, if effective, may have potential to influence the way that evidence-based treatments are delivered in community mental health settings. To this end, initial exploratory research studies are needed that direct the process of developing such resources and that examine their potential to improve the quality of mental health care throughout the course of treatment. Findings from the present study highlight trainers’ perceptions that technology may be a valuable resource to facilitate treatment delivery if used properly, and that such resources may have potential to enhance provider fidelity and competence and increase patient engagement.