Introduction

The growth of the evidence-based practice (EBP) movement has led to an increased need for linkages between research and clinical practice. Transferring evidence-based treatments of borderline personality disorder (BPD) to community-based settings is a common goal among stakeholders in public behavioral health systems given the disproportionate amount of behavioral health services used by patients with BPD and the subsequent financial burden placed on systems of care (Comtois et al. 2007; Linehan and Heard 1999). Dialectical behavior therapy (DBT) is an evidence-based treatment that is the standard of care in treating BPD (Linehan 1993). Eleven controlled trials of DBT have shown to be efficacious in reducing suicidal behavior, inpatient admissions, and other key outcomes (Bohus et al. 2013; Koons et al. 2001; Linehan et al. 1991, 2006, 2002; Lynch et al. 2003; Telch et al. 2001; Verheul et al. 2003). The majority of research on the efficacy of DBT has been conducted within tightly-controlled research lab settings, and similar to many other EBPs, there are far fewer published studies that evaluate the effectiveness of DBT in diverse clinical settings with less stringent inclusion criteria to maximize generalizability of findings (Ben-Porath et al. 2004; Comtois et al. 2007; Kazdin 2008).

There is a paucity of research to develop strategies for implementing DBT in public behavioral health systems where there is often a lack of resources and infrastructure to develop and sustain EBPs (Hawkins and Sinha 1998). Additional effectiveness and implementation research studies are needed to determine the barriers and solutions to implementing DBT within health systems to ensure that a larger and more diverse clinical population can benefit from receiving evidence-based care (Ben-Porath et al. 2004; Chambless and Hollon 1998; Koerner 2013).

There are several DBT effectiveness studies that have taken place within public mental health settings and have shown promising results. Pasieczny and Connor’s (2011) evaluation of DBT within a public mental health system in Australia demonstrated that DBT showed significant reductions in suicidal and non-suicidal self-injury, emergency department visits, psychiatric admissions and inpatient bed days, compared to treatment-as-usual (TAU). The effects of training amount on patient outcomes was evaluated using a between groups analyses of clinicians who received 4 days of DBT training versus those who attending an intensive DBT training. Patients with BPD who received treatment from clinicians attending the intensive DBT training showed significantly lower rates of suicidal behavior compared to the group of patients receiving treatment from clinicians with less DBT training. Another recent evaluation of DBT found that compared to TAU, patients with BPD has significantly reduced self-harm behaviors (Priebe et al. 2012).

Turner’s (2000) naturalistic study of DBT included a true-experimental design that demonstrated the efficacy of DBT in the treatment of 24 patients with BPD in a community-based setting. Participants receiving DBT showed a decrease in suicidal behavior and the rate of psychiatric hospitalization utilization, compared to the patient-centered therapy control condition. Ben-Porath et al. (2004) conducted a community-based evaluation of DBT in the treatment of patients with BPD that were high utilizers of psychiatric services, and included no diagnostic exclusion criteria. The results indicated that DBT resulted in a significant reduction in suicidal behaviors as well as a significant decrease in therapy-interfering behaviors including treatment non-compliance and lack of attendance. Trupin et al. (2002) evaluated the effectiveness of DBT in treating incarcerated female juvenile offenders and reported mixed results on behavior change among the patient sample, however a significant decrease was found in the staff’s use of restrictive and punitive actions (i.e., room confinement).

Comtois et al. (2007) conducted an evaluation of DBT in the treatment of 23 patients with BPD that received outpatient treatment in an urban community mental health agency, with no exclusion criteria based on comorbid diagnoses, such as bipolar mood disorder or a psychotic disorder, thus maximizing generalizability of the results to the various settings where multi-diagnostic patients are likely to receive care. Receiving DBT resulted in significantly reduced rates of psychiatric-related emergency room visits, psychiatric inpatient admissions, crisis-related inpatient admissions, psychiatric-related inpatient days, and number of crisis services engaged (Comtois et al. 2007).

While many of these studies have demonstrated the effectiveness of DBT, there are limited published implementation research studies on DBT that have examined models of training. One of the only published implementation studies on this topic, by Swales et al. (2012), evaluated the sustainability of multiple DBT programs after receiving intensive DBT training as part of a large roll-out within healthcare settings in the United Kingdom. Programs were particularly vulnerable to drift at a point in the second year of a training program, and several prevention measures were identified including assessing whether the intervention matches the organizational and service needs, careful selection and training of staff and of appropriate clinical populations, maximizing the use of the time of clinicians, and determining that there are systems put in place to monitor the programs and address barriers in order to improve outcomes at both the clinical and a program level.

While there have been no published controlled training evaluations of DBT, there is data to suggest that community-based clinicians are trainable in DBT (American Psychiatric Association 1998; DuBose and Ivanoff 2013; Hawkins and Sinha 1998; Landes and Linehan 2012). More studies are needed to examine methods of implementing DBT, including identifying efficient models of training, developing strategies for preventing drift, and engaging clinicians to identify the limitations and strengths of implementation efforts in diverse clinical settings.

Previous studies on the implementation of EBPs have identified a need for researchers to incorporate the perspectives of the administrative as well as clinical staff, as this can be helpful in identifying the factors that lead to successful implementation of an EBP (Bloch et al. 2006; Gray et al. 2007). McHugh and Barlow (2010) stress the importance of examining levels of engagement following EBP training, and in gathering information on individual clinician performance and participation in reaching competence standards as a way of determining the models for effective implementation. Collecting information on the perspectives of stakeholders within evaluations of EBTs can also provide key insights into the challenges of training, setting, and evaluation of adherence to fidelity and patient outcomes (Gotham 2006).

There is no published study to our knowledge that examines the procedures of a system-wide roll out of DBT in a public sector setting in the US, nor any study that engages the perspectives of clinicians within multiple community mental health and substance abuse settings to identify the barriers and solutions to implementing DBT. The following study examines the challenges of implementing DBT in a large public behavioral health system. A 10 day comprehensive DBT training (80 h) was provided over a period of 13 months by an expert trainer in DBT. The training sessions occurred over a 1–2 year period, thus differing from the standard 10 days intensive model of DBT training. A variety of resources on maintaining successful adherence to DBT were made available to clinicians including access to a DBT listserv, an online forum of video demonstrations of DBT interventions, recordings of each training session after they were administered, phone consultation, and feedback on recorded sessions.

The first training provided an introduction to DBT, and included background of the biosocial theory and an overview of the research supporting the efficacy of DBT. A 3 days training followed 1 month later, and focused on program development and intensive team building. At 4 months post baseline, a 2 day training was presented on DBT skills, problem assessment, skills coaching and DBT consultation team building. An advanced training in DBT followed 13 months after the initial training and focused on review of previous materials and case consultation. At least one case example was presented during each training session, which consisted of discussing the case and identifying primary targets, doing a role-play of interventions, and providing follow-up consultation. Trainings that occurred later included more time for case examples and at times included up to three case examples per training session. A train-the-trainer model was chosen as a way to prevent drift, however it should be noted that this model failed to be implemented due to the inability of full-time clinicians to incorporate non-clinical hours into their work contracts to provide the training.

The goals of the current study are to determine (1) what are the barriers and solutions to implementing DBT in mental health and substance abuse settings within a public behavioral health system; and (2) how can models of training and implementation effectively address these identifiable barriers.

Method

Participants

This study included a sample of clinicians in community mental health and substance abuse agencies within a public behavioral health system in Northern California. A total of 34 clinicians took part in the training and provided a range of services including substance abuse outpatient treatment, intensive case management and/or outpatient mental health care. DBT training was provided as part of a system-wide performance improvement project to increase the capacity of clinicians to effectively treat patients with BPD, and agencies volunteered to participate in the training and evaluation project. Structured phone interviews were conducted with 19 of the 34 clinicians who had received DBT training in order to identify challenges to the DBT implementation process. Fifteen of the clinicians were not interviewed because they either declined participation or were no longer employed in the same clinical role as when they attended the training. Demographic data was collected from all 34 clinicians at the onset of DBT training. However, this information did not contain identifiers, so it was not possible to extract the demographic information for the 19 qualitative interview participants. Therefore, the following data on sex, age, clinical experience and educational levels is reported on the full sample of training attendees. The clinicians were 88 % female, and had a mean age of 39 years (SD = 9.62), a mean of 8.7 years (SD = 8.82) of clinical experience, and a mean of 1.2 years (SD = 3.16) of previous DBT experience. The highest educational levels of the clinicians included 94.1 % (N = 32) with a masters degree, 2.9 % (N = 1) with an associates degree, and 2.9 % (N = 1) with a doctoral degree.

Procedures

Qualitative data was collected during a series of interviews with 19 DBT clinicians that received DBT training. The interviews contained a structured set of items including whether participants have received adequate training in DBT, feedback on what to include in future trainings, challenges to implementing DBT and ideas for how to address the challenges, as well as information on their clinical activities including their participation in the four components of the DBT model (individual therapy, skills training, phone coaching and consultation team meetings). Analysis of interview transcripts was conducted using a content analysis approach including identifying data codes and transforming them into categorical themes, and subsequently sorting the themes to identify meaningful patterns. Preliminary categories were created based on observation of the major themes and ideas within the responses and descriptive statistics were analyzed based on the frequencies of responses.

Results

Respondents were asked to identify the challenges of implementing DBT in their particular agency. Several clusters emerged based on the commonalities of the responses, including three themes that were identified as challenges to the DBT implementation process including (1) Program development and recruitment of patients, (2) lack of administrative support or organizational investment in DBT, and (3) time commitment of DBT (see Table 1).

Table 1 Summary of major themes and ideas regarding DBT implementation efforts
Full size table

Theme 1

Theme 1 included programmatic challenges of developing a DBT program, maintaining staffing and recruiting patients that were appropriate for DBT, and 47 % (N = 9) of the respondents reported such challenges. Clinician comments within this theme reflected the fact that DBT requires a significant amount of training and that staff turnover, or insufficient numbers of staff to begin with, can jeopardize the sustainability of a DBT program. As one participant noted, “Our biggest challenge is staff cutbacks from budget cuts. We don’t have enough people trained in DBT to have regular group facilitators with back-ups.”

Of the 19 clinicians that were interviewed, three clinicians represented one clinical team that experienced significant problems developing their DBT program within an outpatient substance abuse treatment facility providing methadone maintenance for opiate-dependent patients. One of the three respondents on the team noted the primary challenge of starting the DBT program was the recruitment and engagement of patients, and that while the agency did serve a high proportion of patients with BPD, the obstacle was that patients would often present at the clinic with the sole purpose of obtaining their dose rather than to receive psychotherapy services. DBT and other intensive psychotherapy treatment programs were seen as incredibly useful skills for the clinicians to obtain, yet ultimately the practice of these treatments was incompatible with the structure of the substance abuse treatment facility that offered primarily short-visits for methadone dosing rather than long-term visits to receive individual or group psychotherapy.

Barriers associated with program development and patient recruitment can be addressed by conducting site-specific assessments to determine whether to adopt DBT given constraints and demands of particular programs. Themes to consider in this assessment include the time commitment of clinicians, the level of support of administrators in providing evidence-based care to patients with BPD, and availability of feasible training options. The findings of this study support the idea that complex and intensive treatments such as DBT require ongoing consultation in order to achieve sustainability. Collaboration between teams at different agencies within a public behavioral health system can offer opportunities to share didactic training and merging DBT consultation teams.

Theme 2

Theme 2 was identified by 42 % (N = 8) of the sample, and included a lack of administrative support or investment in DBT. Clinicians felt that a major barrier to implementing DBT was lack of support from clinic management, which included prioritizing other treatment programs within the agency over DBT. Limited interest in providing evidence-based care to patients with BPD was also included in this theme and in some cases included pejorative attitudes towards patients with BPD. This was evidenced by a respondent stating, “My director hates BPD patients and wants them out of the program. I think it’s a lack of understanding of BPD and DBT on the [the director’s] part…it would be helpful to explain why people would be interested in doing work with BPD.”

Providing trainings or in-service presentations to administration as well as clinical staff were identified as solutions to the negative attitudes toward BPD and/or the lack of understanding of DBT that was present among several clinical administrators and clinical staff. It should be noted that the pejorative views towards patients with BPD among staff was considered by more than one respondent to be a factor that impeded access to care for patients, and it was thought that information presented in a training or in-service would have the effect of reducing negative attitudes that were based in a lack of knowledge about the disorder. A strategy to compensate for a lack of investment in DBT was to create more communication and support across DBT teams operating within different agencies. Merging consultation teams was one method of sharing clinical time and resources among DBT programs that had limited infrastructure or administrative support.

Theme 3

The third theme that emerged was the time commitment of DBT and the failure to reduce other clinical responsibilities in order to account for a clinician’s involvement in DBT. A total of 42 % (N = 8) of the participants reported this concern, as one respondent stated, “Because there’s so few of us [on the DBT team] and we have a huge caseload, [administrators] say we’re not supposed to have as big of a caseload as everybody else, but that doesn’t happen.” Solutions to this barrier included prospectively determining clinical time for DBT and maximizing the use of the time of skilled DBT clinicians, similar to the strategies outlined by Swales et al. (2012).

Results of Training Evaluation

Regarding the adequacy of DBT training in general, all but one participant reported that their training was satisfactory (one respondent who did not attend any of the trainings reported that the DBT training was unsatisfactory). When asked about their feedback for future trainings, respondents requested more case examples with more detailed instructions on how to do specific interventions, such as conducting a chain analysis. One respondent remarked that “more consultation about specific cases would be helpful” and that “case consultation brings the concepts into real-life situations.” Additionally, many respondents wanted to “go over specific techniques and concepts that are used in both individual and group” and had hoped to review this process with the trainer directly through the use of more Q&A sessions and demonstration of using techniques during the trainings.

The respondents were asked to provide general feedback on how trainings can address the challenges of implementing DBT mentioned above. Several respondents discussed the difficulties in establishing collaboration between teams at different agencies and viewed this collaboration as key to sustainability of their DBT program, due in part, to the changing of staffing and the loss of many team members due to financial cutbacks. Sharing resources, such as merging consultation teams or sharing training materials, was seen as an important step to keeping their DBT program running. Attending a system-wide training on an ongoing basis served as a way for smaller teams to communicate with teams that were more well-established and to share resources. Several participants appreciated the value of training in offering a gathering of community providers to address these barriers, when otherwise there would be minimal communication between the teams.

Discussion

An analysis of 19 qualitative interviews was conducted using a content analysis approach to identify common themes regarding the barriers to implementing DBT in a public behavioral health system and solutions to improving DBT training and consultation to address these barriers. This is the only published study to date to examine the process of implementing a system-wide roll out of DBT in a public behavioral health system in the US, with the goal of engaging clinicians working in both mental health and substance abuse settings to better understand the barriers associated with implementing DBT. The study had several important limitations including selecting agencies who sought out DBT. Because agencies volunteered to participate in a performance improvement project and receive DBT training, there is likely an overrepresentation of clinicians who were more motivated to pursue training in DBT and therefore not representative of all community behavioral health programs.

Another important limitation was differing data collection procedures for demographic variables and qualitative interview data, which prevented the research team from establishing differences between participants who did and did not complete the interviews. Additionally, there was no system of tracking participants’ attendance across all trainings other than self-report.

Events occurring concurrently with the study added threats to the validity of findings, including that within the 13 months of training, the public behavioral health system initiated layoffs and financial cutbacks that resulted in considerable layoffs among clinical teams. This should be taken into account given that agency support was identified as a major obstacle for implementing DBT. Attrition was another limitation of the study as evidenced by the fact that 15 of the 34 clinicians who attended the first training were not interviewed because they either declined participation, were no longer employed in the same clinical role, or were no longer employed within the health system.

The results indicated that all respondents who attended the DBT trainings thought that they received adequate training in DBT. There was no mention of clinical resources outside of in-person training that were made available to all teams, including access to a DBT listserv, an online forum of video demonstrations of DBT interventions, recordings of all previous trainings, as well as available phone consultation and feedback on recorded sessions from the trainer. These tools were provided to help prevent drift, yet no respondent referenced these tools during the interviews nor did any respondent follow-up to receive consultation or to request a session review. This suggests that these resources were not distributed adequately, or that the primary in-person trainings were valued more than any tool provided to supplement them. This is consistent with the findings that the respondents prefer more clinical examples and more detailed instructions on how to perform specific interventions through the use of case examples and role-plays. Future implementation research studies should examine whether online, phone or other low-cost training delivery methods can as effectively as in-person training. Component analyses of EBP trainings are needed to determine the specific therapist-patient interactions in DBT that effect change, as noted by Koerner (2013).

A lack of administrative support and/or overall agency support was identified as the most common challenge to implementing DBT. These findings illustrate the importance of investment from both clinic-level and system-level administrators in the process of implementing EBPs starting with the decision of whether to adopt an EBP. A thorough needs-assessment is required in order to determine if adopting DBT fits the goals and the capacity of the program. The previous example of an outpatient opiate replacement therapy clinic that was unable to develop their DBT program is an example of the many pitfalls that can occur when an adoption of an EBP takes place in the absence of a sufficient needs-assessment. Many programs in the outpatient substance abuse field, in particular, would have to undergo significant structural changes in order to implement an intensive psychotherapy program like DBT, therefore potential barriers such as these should be considered before DBT is adopted. In the current study, organizations with an infrastructure and framework to support intensive outpatient treatment programs (with outpatient programs such as intensive case management already in place) did not report similar challenges of recruiting and engaging patients; rather many of these organizations reported barriers that included a lack of a time commitment for DBT. Site-specific needs assessments should focus on whether to adopt DBT given the constraints and demands of particular clinics. Considerations should include the time commitment of clinicians, the full support of administrators in providing evidence-based care to patients with BPD, and availability of feasible training options that emphasize case examples and demonstration of DBT interventions.

The results of the current study reflect the understanding that ongoing consultation is necessary in order to develop and sustain a DBT program. Collaboration between teams at different agencies within a public behavioral health system can offer opportunities to share didactic training and merging DBT consultation teams, and a train-the-trainer model is another method of obtaining ongoing consultation and preventing drift. In order to implement the train-the-trainer model, it is essential to determine in advance whether full-time clinicians have the capacity to perform training activities within their job role.

The absence of studies with rigorous methodology for evaluating EBP training models should be addressed in future research in order to aid our understanding of how to deliver complex and sophisticated treatments such as DBT. Component analyses of training models and practice-based training research strategies are needed in order to maximize effective and efficient DBT training. Engaging providers in an understanding of their perspectives on the barriers and solutions to adopting DBT is a recommended implementation strategy, as these barriers will vary based on the needs and infrastructure of programs in any given agency or health system.