Introduction

Annual rates of staff turnover (i.e., the separation of an employee from an organization)1 in community behavioral health agencies are consistently reported to be between 30 and 60%.2,3,4,5 These rates are far greater than the 10% annual turnover rate that is considered healthy for organizations.6 A 10% annual turnover rate allows organizations to replace employees, often with more motivated or productive employees, without considerable financial burden.6,7 These financial burdens associated with recruiting, hiring, and training new staff8 are a common problem in organizations with high turnover. There are also other negative outcomes for staff who remain following a period of high turnover, including poor morale, diminished productivity9, and reduced quality of services.10,11 Within behavioral health, staff turnover is particularly problematic because when clinicians leave, clients must re-engage with a new clinician, which can be difficult. Perhaps, this is partially why associations have been found between staff turnover and negative client outcomes.12

Given the problems associated with high rates of turnover, it has been the subject of a substantial body of literature, which has evolved in various ways over the past few decades. Research efforts largely began by identifying predictors of turnover within the workforce as a whole, rather than by specific industry. The most recent meta-analysis of the general workforce turnover literature noted that a greater number of children, a shorter tenure with the organization, weak organizational commitment, poor leadership, low perceived autonomy, and low job satisfaction were all associated with higher rates of turnover.13 More recently, efforts have focused on specific industries, given the inherent differences in job stressors and work environments across industry types. This is particularly important for behavioral health professionals, given that they are more susceptible to burnout (i.e., the combination of emotional and/or physical exhaustion, diminished productivity, and depersonalization)14 and job stress (predictors of turnover for this population)15,16,17 than the general workforce.18 Although numerous predictors of behavioral health turnover have been noted, among the most frequently and consistently reported predictors3,15,17,19 are organizational culture (i.e., “normative beliefs and shared behavioral expectations” regarding how things are done in a work unit)10 and organizational climate (i.e., employee perceptions of the overall work environment).20

More recently, perhaps due in part to the increased call for the implementation of evidence-based practices (EBPs),21,22 focus has turned toward understanding factors that may contribute to turnover for behavioral health professionals specifically involved in the delivery and/or implementation of EBPs. Turnover is especially problematic in organizations that are involved in the implementation of EBPs, as the preparation, training, and supervision required for successful implementation are costly.23 Once clinicians trained in an EBP leave their organization, that organization is no longer able to offer the EBP,24 resulting in a poor return on investment for the organization.

Although turnover is well-researched, critical gaps remain in the literature, particularly relating to turnover within the context of EBP implementation. First, EBP training is considered an essential component for implementation and sustainability of the intervention.25,26 A review of the EBP training literature identified the three most common training methods (learning collaborative, cascading model, and distance education), all of which vary considerably in terms of the burdens and supports for trainees.27 As role stress and organizational support have been identified as predictors of turnover,15,17, it is important to consider that the added stress of EBP training, in absence of organizational support, may contribute to turnover. However, implementation studies that report on provider turnover have used one training method for all providers within the study, and thus, it has not been possible to examine the extent to which different training methods might impact turnover.

The second critical gap within the literature concerns the type of staff member included as a study participant. With a few exceptions, most studies examining predictors of turnover for organizations implementing EBPs2,4,17 and those examining training outcomes27 have only considered direct service providers (e.g., clinicians, therapists, case workers). This focus has occurred for a number of important reasons. First, EBP training efforts often only involve direct service providers, with the noted exception of learning collaborative training models. Second, the implementation components responsible for high costs (e.g., training, supervision)23 also only involve direct service providers. Third, by definition as direct service providers, they are responsible for client outcomes and are at a higher risk for burnout due to regular interaction with challenging cases.28 However, this focus on direct service providers is a limitation within both the turnover literature and the training literature, given that poor organizational and administrative support is often implicated as both a predictor of greater clinician turnover16 and as a barrier to EBP implementation.3,29 As such, it is of critical importance to consider staff members other than direct service providers (e.g., supervisors, administrators), as they may experience and contribute to turnover in ways not previously explored.

The aim of the current study is to address these limitations in order to better understand staff turnover on a number of levels. First, the current study includes three different EBP training models, which will allow for an analysis of differences in turnover across these training models. It was hypothesized that rates of staff turnover would be higher in the distance education model, as this model provides the least amount of organizational support for trainees. It was also hypothesized that staff turnover would be lowest in the learning collaborative model, which involves the greatest amount of organizational support. Second, the current study will examine differences in the rates of turnover for clinicians, supervisors, and administrators. As some studies have noted lower rates of turnover supervisors or administrators than for clinicians,3,11, it was hypothesized that rates of clinician turnover would be greater than rates for supervisors or administrators. Third, the current study will look at predictors of turnover for clinicians, supervisors, and administrators. Given the literature on predictors of turnover for behavioral health providers, it was hypothesized that poor organizational culture and organizational climate would be predictors of turnover for all three groups.

Method

Setting

Data for the current study was collected as part of a larger federally-funded study designed to evaluate the effectiveness of three different training models during the state-wide implementation of Parent-Child Interaction Therapy (PCIT; NIMH R01 MH095750). IRB approval was obtained through the participating institution. In the interest of brevity, method and procedural details that are relevant to the current study are provided. For additional information on method and procedure details of the parent study, interested readers are invited to reference the study protocol.30

PCIT is a manualized behavioral parent training program and is considered an EBP for children ages 2.5 to 7 years with disruptive behavior disorders or with a history of maltreatment.31 Caregivers and children participate in treatment together, typically for 12 to 20 weekly, 1-hour, outpatient sessions. PCIT includes two phases of treatment; the first phase focuses on relationship enhancement, and the second phase focuses on effective discipline strategies to improve child compliance.31 The use of in vivo coaching through a bug-in-the ear device and one-way mirror makes PCIT unique from many other behavioral parent training programs. Although PCIT was originally developed to target externalizing behavior problems, it has been adapted to treat a variety of child mental health concerns. PCIT has been the focus of a substantive body of research.32

Participants

Outpatient clinics and their staff members were recruited by research team members. The larger parent study took place across the state of Pennsylvania (PA). County administrators for all 67 counties in PA were approached about the study, and 40 agreed to participate in informational meetings. Clinics within those 40 counties were eligible to participate in the study if they met the following criteria: (a) psychiatric outpatient clinic licensure in PA, (b) willing to participate in PCIT training, (c) the ability to cover site preparation costs, and (d) agreeable to research participation.30 Clinic administrators were defined as an executive director, chief financial officer, or other individual responsible for daily operations at an enrolled clinic; there were no other inclusion criteria for administrators.30 Supervisors were eligible to participate if they were employed at an enrolled agency, had been identified by the administrator as the program lead, and were willing to participate in training if they were assigned to the learning collaborative condition.30 Clinician inclusion criteria were as follows: (a) employment at an agency that had elected to participate in PCIT training, (b) masters or doctoral degree in a human services field and current licensure or receiving supervision from a licensed individual, (c) a current caseload that included clients appropriate for PCIT, (d) receptive to receiving PCIT training (e) had not been previously trained, and (f) willing to complete research-related tasks.30 Ultimately, 100 clinicians, 50 supervisors, and 50 administrators from 50 clinics agreed to participate.

Procedures

All participants completed a battery of assessments at four time points: baseline, 6- (mid), 12- (post), and 24-months. If a member of the research team learned of a staff member leaving the clinic at any point outside of the assessment windows, the team member followed up with the participant to complete the Agency Staff Change Form (described below). This was to ensure accurate reporting of turnover rates and to measure variables associated with turnover as close to the time of turnover as possible. The majority of clinicians and supervisors (86%) completed questionnaires online. Participants without internet access were given the option of completing questionnaires over the phone or on paper. The remaining 14% of clinicians and supervisors who did not complete questionnaires online chose to complete paper copies. Data was collected from all administrators over the phone.

Training Conditions

Cascading Model

The cascading model, also known as “train-the-trainer,” is the training model that has been endorsed by the PCIT International Training Committee.33,34 The initial training consisted of a 5-day (40 hours) face-to-face training with a PCIT International certified trainer, followed by a 2-day (16 hours) face-to-face training 6 months later. Participants also received bi-weekly group phone consultation for 12-months (24, 1-hour calls over 1 year). Afterward, clinicians participated in an additional 6 months of consultation and training focused on training others within their agencies.30

Cascading model trainings require substantial up-front investment, with considerable time required to attend the initial training as well as the ongoing consultation. However, the primary benefit of a cascading model is that it allows trained clinicians to return to their agencies and function as the trainer for other clinicians, with the intention of promoting more successful sustainability without any effort needed by higher-ranking staff members (e.g., supervisors or administrators). Research indicates that workshop trainings with ongoing follow-up, such as cascading models, are effective in promoting clinician behavior change including the use of newly learned skills.27

Learning Collaborative

The learning collaborative model takes a clinic-based approach to EBP implementation and involves specialized training sessions for clinicians, supervisors, and administrators. Within the current study, two clinicians, one supervisor, and one administrator from each clinic participated in the learning collaborative. Based on recommendations from the National Child Traumatic Stress Network, which routinely implements learning collaborative trainings,35 the learning collaborative condition included three phases: pre-work, learning sessions, and action periods. The 3-month pre-work phase consisted of a review of readings and materials, and conference calls with PCIT experts. Learning sessions were carried out over a 9-month period and consisted of three, 2-day face-to-face meetings. Action periods occurred between learning sessions and incorporated the use of improvement data, technology, team meetings, and conference calls to support learning. Like the cascading model, participants were eligible for PCIT certification following the initial 12-month training period. The intent of the learning collaborative model is to provide the entire organization with the support and resources needed to promote the long-term sustainability of the intervention. However, there is mixed evidence regarding its ability to promote clinician behavior change and use of the intervention.27

Distance Education

Distance education generally refers to a training model in which trainees learn the material at their own pace away from a traditional, face-to-face setting. An online course developed by the PCIT Team at the University of California, Davis (SAMHSA grant; PI: Urquiza) was used for the distance education condition. The training course included 11 modules incorporating written materials, vignettes, videos, and quizzes; the entire training took clinicians approximately 10 hours to complete.30 Consistent with the other two training models, each clinician in this condition was provided with the PCIT manual, the Dyadic Parent-Child Interaction Coding System (DPICS) Manual, and the DPICS workbook.30 In addition, participants completed phone consultation with a trainer and were eligible for PCIT certification after 12 months, consistent with the other training conditions.

Advantages to the distance education condition include that it is free and is the least time-intensive. However, results of the few studies that have examined outcomes of distance education trainings indicate that they are not often associated with changes in clinician behavior or EBP use.27 Thus, while the distance education condition might be convenient to complete, it is unclear whether it produces the desired increase in clinician knowledge, skill, and EBP use.

Measures

Demographics

Demographic information was collected from clinicians, supervisors, and administrators using the Background and Contact Information Form, which included standard demographic information (e.g., gender, race, education level) and information regarding the respondent’s current role, such as the amount of time employed by the agency and the amount of experience within the human services industry. Demographic information was only collected during the baseline assessment.

Agency Staff Change Form

The Agency Staff Change Form contained questions related to changes in employment. These forms were completed by clinicians, supervisors, and administrators at all four timepoints. As previously mentioned, study team members also completed the form if they learned about a staff member leaving the original clinic at any point during the study outside of assessment windows. Both the Agency Staff Change Form and the Background and Contact Information Form have been used in previous implementation trials.36

Organizational Readiness for Change (ORC)

The Organizational Readiness for Change (ORC) questionnaire37 is a 115-item self-report questionnaire completed by supervisors and administrators in the current study. This questionnaire was designed as a comprehensive assessment of an organization’s overall functioning and readiness for change. When completing the ORC, participants rated their level of agreement with each item on a 5-point Likert-type scale (1—disagree strongly, 2—disagree, 3—uncertain, 4—agree, 5—agree strongly). The ORC includes four main scales. Included in the current study were the organizational climate and program resources scales. The items on these scales map onto the constructs of organizational climate and culture (respectively) that have been found to predict staff turnover. Scores on these scales range from 10 to 50, with higher scores indicating more positive ratings of climate or culture.

Research has indicated that responses on the ORC have adequate psychometric properties. Cronbach’s alpha was computed for each of the subscales as an estimate of internal consistency and was adequate (above .70) for the majority of the subscales.36 Responses from the current sample of supervisors and administrators indicated adequate internal consistency for the organizational climate (α = 0.81) and program resources (α = 0.68) scales.

Survey of Organizational Functioning

The Survey of Organizational Functioning (SOF) is a 162-item self-report questionnaire that was developed based on the ORC and was completed by clinicians in the current study. Participants rated their level of agreement with each item on a 5-point Likert-type scale (same responses as ORC). Because the SOF is geared toward clinicians, there are more items than on the ORC and seven main scales. As with the sample of supervisors and administrators, the organizational climate and resources scales were used as indicators of organizational climate and organizational culture, respectively. Scores on these scales range from 10 to 50, with higher scores indicating more positive ratings of the climate or culture. Although no published reports on the psychometrics of the SOF currently exist, results from the current sample of clinicians indicated acceptable internal consistency for both the organizational climate (α = 0.83) and the resources (α = 0.68) scales.

Definition of Turnover

Turnover as assessed within the current study was defined as an employee separating from the original agency. Employees who changed roles and/or positions within the same agency were not included in the turnover count. Additionally, the current study differentiated between voluntary and involuntary turnover and only included participants who voluntarily left their agencies. Finally, turnover rates were calculated for both the 12-month training period and for the entire 24-month study duration.

Data Analyses

All analyses were conducted either in Statistical Package for the Social Sciences (SPSS version 24.0)38 or in HLM, version 7.39 Turnover rates were calculated for 12 and 24 months. A series of chi-square tests of independence were used to determine if turnover rates differed by job type or by training condition.

In order to examine predictors of turnover, supervisors and administrators were combined to form one sample. This decision was made given their different day-to-day responsibilities compared with clinicians, their shared measure of organizational climate and culture (i.e., the ORC), their similar roles within each training condition, and to maximize power. Given the nested structure of the data (staff members within agencies), a series of hierarchical linear modeling (HLM) analysis was used to assess for significant organizational predictors of (a) clinician and (b) supervisor/administrator turnover.

Results

Sample Characteristics

Table 1 contains the full demographic information for clinicians, supervisors, and administrators. Clinicians (n = 100) were primarily Caucasian (91%) and female (84%) and were an average of 39 years old (SD = 10.04) at baseline. The majority (92%) held a master’s degree in either in psychology (37%) or social work (31%), and half (52%) were professionally licensed. Most clinicians were employed full-time (74%) and reported an average yearly salary of $43,939 (SD = $12,712), while those who were employed part time (28%) reported an average hourly wage of $28.55 (SD = $6.77). Clinicians had an average of 11.42 years (SD = 8.20) of experience within the human service industry and an average of 4.82 years (SD = 5.61) at their current agencies.

Table 1 Demographics for clinicians, supervisors, and administrators
Full size table

Supervisors (n = 50) were also primarily Caucasian (88%) and female (80%) and were an average of 45 years old (SD = 9.48) at baseline. Similar to the clinicians, the majority of supervisors held a master’s degree (82%), while the remainder held a doctoral degree. Most held their degrees in either psychology (32%) or social work (36%). Most supervisors (86%) were employed full-time and reported an average yearly salary of $55,991 (SD = $11,132), while those who were employed part-time (n = 5, < 1%) reported an average hourly wage of $29.40 (SD = $4.93). Supervisors had an average of 18.26 years (SD = 8.60) experience within the human service industry and an average of 7.20 years (SD = 5.45) years within their current agencies.

As with clinicians and supervisors, administrators (n = 50) were primarily Caucasian (90%) and female (62%) and were an average of 48 years old (SD = 8.96) at baseline. The majority of administrators held a master’s degree (68%) or a doctoral degree (20%). Most administrators had a degree in social work (32%), with a substantial number holding degrees in psychology (22%) or another field (30%). Administrators in the current sample reported an average of 22 years (SD = 8.53) experience in the human service industry and had worked an average of 12 years (SD = 8.40) at their current agencies.

Rates of Turnover

In order to maintain a clear picture of turnover, two different rates of turnover were calculated: one from the time of participant enrollment to the 12-month assessment and one from the time of participant enrollment through the entire 24-month duration of the study. Within 12 months, 11% of clinicians, 6% of supervisors, and 4% of administrators left their respective agencies. By the end of the 24-month study duration, 31% of clinicians, 30% of supervisors, and 26% of administrators had left their respective agencies (Table 2).

Table 2 Rates of turnover
Full size table

Two chi-square tests of independence were run to test the hypothesis that supervisors and administrators would have lower rates of turnover (at 12 and 24 months) than clinicians. Contrary to hypotheses, results of the chi-square test indicated that there were no differences in the percentage of clinicians, supervisors, or administrators who left during the first year of the study (χ2 [2, n = 196] = 2.86, p = .24, Cramer’s V = .12) or over the 24-month course of the study (χ2 [2, n = 191] = 0.46, p = .79, Cramer’s V = .05; see Table 2).

A second series of chi-square tests of independence were run to determine if there were different rates of clinician, supervisor, and administrator turnover based on training condition. Contrary to hypotheses, results indicated that there were no differences in the rates of turnover for clinicians in each training condition at the 12-month time point (χ2 [2, n = 96] = 2.10, p = .35, Cramer’s V = .15) or by the end of the study (χ2 [2, n = 95] = 0.51, p = .77, Cramer’s V = .07). Additionally, there were no significant differences in the 12-month rates of turnover for supervisors (χ2 [2, n = 50] = 2.02, p = .36, Cramer’s V = .20) or administrators (χ2 [2, n = 50] = 0.98, p = .61, Cramer’s V = .14). Differences in 24-month turnover rates based on training condition for supervisors (χ2 [2, n = 48] = 0.51, p = .08, Cramer’s V = .32) and administrators (χ2 [2, n = 48] = 4.75, p = .09, Cramer’s V = .31) were not significant, but yielded moderate effect sizes and likely would have reached significance with a larger sample size and greater power. For both supervisors and administrators, rates of turnover were greater in the learning collaborative condition than in the cascading model or distance education conditions (Table 3).

Table 3 Rates of turnover by training condition
Full size table

Predictors of Turnover

Preliminary Analyses

Before HLM analyses were computed, an unconditional model was run to test for the amount of variance in turnover that could be accounted for by nesting. Two unconditioned models were run, one to understand the effect of nesting on clinician turnover, and one to understand the effect of nesting on supervisor and administrator turnover. Results indicated that 18% of the variance in clinician turnover was accounted for at the agency level, while 79% of the variance in supervisor and administrator turnover was accounted for at the agency level. Both ICCs indicate that a significant amount of variance in turnover is accounted for by agency-level factors and supported the use of HLM.

HLM Analyses

Separate files were created in SPSS for each level of the data and for both the clinician group and the supervisor/administrator group. The level-one data files included each participant’s scores on the measures of organizational culture and climate as well as dummy codes for training condition. Once data files had been created and cleaned, they were imported into the HLM software39 for analyses. Each model was run twice: once without training condition included and once with training condition included.

Results indicated that organizational culture did not predict clinician turnover before (coefficient = − 0.05, SE = 0.08, t = − 0.17, df = 47, p = .53) or after (coefficient = − 0.05, SE = 0.08, t = − 0.63, df = 47, p = .53) training condition was added to the model. Organizational climate was also not a significant predictor of clinician turnover either before (coefficient = − 0.02, SE = 0.07, t = − 0.35, df = 47, p = .72) or after (coefficient = − 0.03, SE = 0.07, t = − 0.37, df = 47, p = .72) taking training condition into account.

Organizational culture was also not a significant predictor of supervisor and administrator turnover before (coefficient = 0.03, SE = 0.08, t = 0.36, df = 47, p = .73) or after (coefficient = 0.03, SE = 0.08, t = − 0.36, df = 45, p = .72) accounting for training condition. However, organizational climate did significantly predict supervisor and administrator turnover both without training condition (coefficient = − 0.14, SE = 0.07, t = − 2.09, df = 47, p = .04) and with training condition accounted for (coefficient = − 0.16, SE = 0.07, t = − 2.20, df = 47, p = .03). See Table 4 for all HLM statistics. These results indicated that supervisors and administrators with more positive perceptions of the workplace were less likely to leave their agencies.

Table 4 Predictors of turnover
Full size table

Discussion

Overall, low rates of annual turnover for all participant types were found in the current study. Although no analyses were run to compare rates in the current study with those reported in the literature, the rates found in this study (12-month turnover rate of 8% and a 24-month turnover rate of 30% for all behavioral health staff combined) appear lower than rates commonly reported in community treatment settings (30–60% per year).4,40,41,42 Contrary to hypotheses, no statistically significant differences were noted in rates of turnover across training conditions. Finally, although no significant predictors of clinician turnover or of supervisor and administrator turnover at 12 months were found, organizational climate was a significant predictor of supervisor and administrator turnover at 24 months. This finding is partially consistent with prior research demonstrating workers with poorer perceptions of organizational climate are more likely to leave their organizations.15,36,43,44.

Lower rates of turnover have been reported in other studies examining clinician turnover within EBP implementation initiatives,3,29,36 which has caused researchers to hypothesize that the use of EBPs might protect against turnover. The lower annual rate of turnover in the current study could provide more evidence to support this protective effect. However, research examining differences in turnover before and after implementing an EBP would provide stronger support for a protective effect, as it would help to demonstrate a temporal relation between EBP implementation and a reduction in turnover, in turn helping to eliminate alternative possible explanations. If such support is found, additional research should address potential underlying mechanisms. One hypothesis is that EBPs provide clinicians with more effective methods to treat their clients, resulting in quicker positive outcomes, fewer adverse events, and reduced clinician burnout and subsequent turnover.43 An alternative is that most EBPs require ongoing support or fidelity monitoring, which may be perceived as extra organizational support.2 Given the mounting evidence that clinicians implementing EBPs have lower rates of turnover, future research should consider investigating the mechanism by which this relation occurs.

Also, worthy of additional consideration is the difference in turnover rates noted at 12 (8%) and 24 months (30%) in the current study. Given that the 12-month timepoint corresponded with the end of the training period, it is possible that clinicians remained at their agencies long enough to meet the criteria for becoming certified in PCIT and left after training completion. Given the high rate of referrals for children with disruptive behaviors45 and the resultant industry value associated with PCIT certification, it is possible that clinicians were more competitive for other employment options after receiving PCIT certification. This hypothesis also is consistent with reports of administrators who participated in a qualitative study to understand barriers to the implementation of dialectical behavior therapy. They mentioned this very phenomena; clinicians stayed through the initial training period, but left after training completion.46 Future research should consider investigating this phenomenon, perhaps through survival analyses to determine if there are different predictors of turnover depending on when clinicians leave agencies.

Clinicians’ motivation to complete training may also explain the lack of difference in clinician turnover across training conditions. Although it was hypothesized that the rate of clinician turnover would vary as a function of organizational support within different training conditions, no such difference was noted. It is possible that clinician motivation to complete training was sufficient to retain clinicians across conditions, as all trainings took the same amount of time to result in PCIT certification. An additional possible explanation for the lack of different turnover rates across training conditions was that all training costs (e.g., cost of registration, and materials) were covered and agencies received a small stipend ($1000) to offset start-up costs.30 Thus, clinicians did not have the burden of advocating to supervisors, other authority figures, or managed care companies to have their cost of training covered. As such, the training experience for clinicians in the study may not be analogous to that of typical community behavioral health providers. Perhaps different rates of turnover would have been noted across training conditions if clinicians had been exposed to some of the burdens that accompany EBP training in typical community treatment settings.

Although there were no significant differences in rates of clinician, supervisor, or administrator turnover across training conditions, moderate effect sizes were noted when comparing supervisor and administrator turnover across training conditions and differences likely would have reached statistical significance with a larger sample. Specifically, by the end of the 24-month study duration, nearly half of the supervisors and administrators in the learning collaborative condition had left their agencies, whereas only 19% of supervisors and 13% of administrators had left in the cascading model condition, and 24% of both supervisors and administrators had left in the distance education condition. This is particularly interesting given that the learning collaborative condition is the only condition that actively involved supervisors and administrators; cascading model and distance education conditions only required clinician participation. While additional research is needed to more fully examine the impact of training on turnover, it is possible that the additional effort (above and beyond typical expectations for supervisors and administrators) that accompanies training may influence turnover. Thus, although the learning collaborative condition is designed to promote long-term sustainability of the intervention through increased organizational support at all levels, it is possible that the extra burden placed on supervisors and administrators contributed to their decisions to leave. This is especially plausible as agencies were randomized into conditions; supervisor and administrator buy-in may have been stronger had agencies been permitted to self-select into the learning collaborative. In addition to the required involvement for supervisors and administrators, the actual activities in learning collaborative sessions, which often focus on administrative issues, may have contributed to turnover. While the goal is to address these issues and to brainstorm preemptive solutions to barriers, the focus on administrative obstacles may have increased dissatisfaction with their agencies, in turn contributing to turnover.

A strength of the current study was that it was one of the first to look at predictors of turnover for supervisors and administrators. Although no significant predictors of clinician turnover were identified in the current study, it is interesting to note that organizational climate did predict turnover for supervisors and administrators. This finding is were partially consistent with prior research.15,17,44 Although results of a follow-up one-way ANOVA assessing for differences in organizational climate based on training condition were not significant, supervisors and administrators in the learning collaborative condition had the lowest average rating of organization climate compared to supervisors and administrators in the other two conditions. Given that supervisors and administrators were not required to participate in training activities in either the distance education or cascading model conditions, it is possible that the added burdens associated with this training condition negatively influenced their perceptions of workplace climate, thus contributing to their decision to leave. Perhaps, the learning collaborative activities designed to help supervisors and administrators preempt administrative barriers lead to more negative perceptions of workplace climate.

However, it is interesting that organizational culture, which corresponds to the day-to-day workplace practices, was not predictive of turnover for supervisors and administrators. As such, the findings within the current sample suggest that self-reported daily workplace practices and responsibilities did not differ for supervisors and administrators based on training condition, but their perceptions of, and attitudes toward, the workplace environment did. It is important to keep in mind that culture and climate are multidimensional constructs. Although a small sample size and limited power precluded a more fine-tuned analysis of the underlying unidimensional indicators, it would be interesting to know if there were particular facets of organizational climate that drove this significant finding.

Taken together, it is clear that future research on turnover within implementation initiatives should consider including all staff members involved in implementation. Although some research has examined the influence of clinician-reported organizational leadership43 and supervisory practices47 on clinician turnover, no studies to date have assessed whether administrator or supervisor reports of their own leadership practices, job stress, or other variables might trickle down and impact clinicians. This may be an important factor to consider in light of the current findings in which organizational climate influenced supervisor and administrator turnover and differences across trainings conditions approached significance, but no predictors of clinician turnover were identified.

Limitations

Despite the strengths of the current study, some limitations are worth noting. Turnover is a complex process that is influenced by numerous interacting variables. As such, complex analyses requiring substantial statistical power are often required to obtain a comprehensive understanding of turnover. Many turnover researchers have moved toward structural equation modeling (SEM) for understanding turnover.15,43,47,48 SEM enables researchers to more precisely model direct and indirect relations between unidimensional and multidimensional predictors of turnover, as well as the crossover effects that can occur when variables are measured at different levels (e.g., individual vs. organizational level variables). Although this type of modeling is ideal for complex processes like turnover, it requires a large sample size that was not available in the current study.

The small sample size within the current study was further limited by the nested structure of the data. Although HLM is designed to account for this nested structure, statistical power within HLM is determined by the number of groups at the highest level—in this case, 50 agencies. It is possible that, despite best efforts to maximize power within the current study, the sample size was too small to identify significant predictors, especially for clinicians.

In addition to quantitative analytic methods, researchers are increasingly turning to the use of mixed methods49 to understand turnover. The subjective nature of many factors that might influence an individual’s decision to leave (e.g., motivations, perceptions of the work environment) may not be adequately captured by self-report surveys often used for quantitative analyses. While the reliance on self-report, quantitative data is a limitation of the current study, future research should consider incorporating qualitative interviews or other qualitative methods in order to obtain a more complete, nuanced understanding of turnover.

The goal of the current study was to understand what factors predicted voluntary turnover, as opposed to involuntary turnover (i.e., employee termination). Methods to capture individual occurrences of turnover were carefully designed to obtain an accurate report of whether turnover was voluntary or involuntary. However, it is possible that human resource practices within some agencies may have precluded the disclosure of employee termination, and some instances of involuntary turnover may have been inadvertently included. This limitation is not unique to the current study and is found in most research on turnover. In fact, some studies have bypassed this limitation by choosing not to distinguish between voluntary and involuntary turnover. However, the differences between voluntary and involuntary turnover were deemed too crucial to ignore in the current study and this limitation was dealt with by attempting to be as accurate as possible in obtaining reports of turnover instances.

The fact that the current sample was primarily Caucasian and female was also a limitation. With such a lack of diversity, caution should be taken in generalizing results to groups with more diverse characteristics. Similarly concerning is the lack of diversity within the behavioral health workforce. For example, in 2004, 51% of licensed clinical psychologists were female and 93% were white, while 82% of clinical social workers were female and 87% were white. Although more recent numbers for specific job types are not available, there is some evidence that the demographics of licensed psychologists have changed recently. Specifically, the percentage of women psychologists has increased to 65% of the workforce, while the percentage of White psychologists has decreased to 84%.50 Despite some evidence that sectors of the behavioral health workforce may be diversifying, the sample obtained in the current study is somewhat representative of the larger behavioral health workforce population (and problem). However, caution should be taken in generalizing the current findings to other regions with greater gender, racial, and ethnic diversity.

Similarly, the current study examined turnover within the context of different training methods for one specific EBP. It is important to keep in mind that various characteristics of the EBP examined in the current study (e.g., targeted population, training requirements, in vivo coaching) are not common characteristics across all EBPs. As such, different results may be found when examining turnover within the context of different EBPs.

One final limitation worth noting is that data used in the current study was part of a larger study not originally designed to assess turnover. However, researchers involved in the parent study design were cognizant of the problematic nature of turnover. Steps were taken to ensure accurate reporting of turnover and to include measures that assess variables related to turnover. Despite these careful considerations, it is possible that important variables were excluded from the parent study, especially as no significant predictors were noted for clinicians.

Implications for Behavioral Health

Results of the current study suggest that training methods should be selected carefully, as the training method used may have an impact on more than just training outcomes such as clinician knowledge, skill, and EBP use. Learning collaborative training models, which were designed to increase organizational support and readiness for implementing an EBP51, may actually backfire if supervisors and administrators are not fully committed to the additional responsibilities required of them or if they become more aware of organizational barriers during the course of training. Individuals involved in the training of behavioral health providers must be aware of such evidence and carefully craft training methods that will provide optimal outcomes across numerous critical domains.

Additionally, organizational climate should be carefully assessed prior to introducing EBP training. As negative organizational climates can result in increased turnover, both for clinicians3,15 and for supervisors and administrators as evidenced by the current study, steps should be taken to remediate these issues before training in order to reduce the likelihood that staff will leave the agency following training. Researchers have recently developed organizational interventions designed to improve the organizational social context and subsequent training attendance.52 As this intervention has been shown to improve the organizational social context (which is a broad domain encompassing organizational culture and climate), this line of research could be extended to examine the effect of organizational interventions on turnover.

Staff turnover is an important problem within the behavioral health field, especially given the recent focus on increasing the number of clinicians trained in EBPs. Results of the current study are consistent with other findings that note a possible protective effect of EBPs on clinician turnover. However, results also indicate that both training method and organizational climate influence turnover rates for staff other than direct service providers. Although turnover is already a complex process, future research should focus on the potential trickle-down effect of job stress affecting administrators, supervisors, and clinicians. Improved understanding of the interrelations between clinician, supervisor, and administrator turnover may help to create training methods that promote improved intervention implementation and sustainability, as well as organizational interventions to increase an agency’s ability to implement and sustain a new intervention.