Clinical supervision is widely believed to be integral to ongoing learning, support, and quality clinical practice with people seeking therapeutic services. The provision of supervision has been a critical component in the education and development of mental health professionals (Borders et al. 2014). Supervisory models to support clinicians in their work with people seeking services abound in the literature (Bernard and Goodyear 2014; Shulman 1993) and knowledge of supervision continues to grow (Bearman et al. 2017; Sewell 2017). In the administration of public mental health services, programmatic service delivery models have included—though rarely explicitly compensate for—clinical supervision and many have required the provision of supervision to staff interacting with people seeking services (e.g., Commission on Accreditation of Rehabilitation Facilities [CARF]) (Hoge et al. 2011). In addition, the growing area of implementation science has pointed to the need for ongoing educational support for providers to translate evidence-based practices into their interactions with people seeking services (Beidas et al. 2013; Gleacher et al. 2011).

Supervision has been significantly related to improving staff and service user outcomes in behavioral health settings (Bambling et al. 2006; Bearman et al. 2013; Henggeler et al. 2002; Mor Barak et al. 2009; Schoenwald et al. 2013, 2009). In clinical evidence-based practice trials, clinical supervision models have been manualized and have improved implementation outcomes such as treatment adherence and fidelity (Henggeler et al. 2002; Henggeler and Schoenwald 1998; Martino et al. 2006; Schoenwald et al. 2009). In some cases, post-training supervisory coaching conducted has been found to be more important than the training quality itself because providers have been able to practice and receive feedback (Salas et al. 2012; Schoenwald et al. 2013; Sholomskas et al. 2005). Elements of evidence-based, best practices in supervision have been articulated in the behavioral health literature and include observation of actual practice (i.e., live supervision with a supervisor providing feedback in vivo), use of direct observation or session recordings (Milne and Reiser 2011), and use of client feedback and outcomes (Worthern and Lambert 2007) to inform the supervision process. Despite the established utility and expectation of supervision, knowledge of best practice supervision availability for providers working in community mental health services remains limited (Dorsey et al. 2018). Evidence for supervision effectiveness is even less robust in non-behavioral health settings (e.g., child welfare) (Carpenter et al. 2014).

In public mental health settings, supervision has been understood to encompass three primary functions of administration, support, and education (Kadushin and Harkness 2002). The hierarchical, casework model of supervision used primarily in these agency settings often utilizes individual, group, or peer supervision modalities and has been sensitive to organizational contextual factors (Bogo and McKnight 2006). Specifically, the service setting type (e.g., school, hospital, or social service agency) has driven availability of supervision along with organizational policies, complexity of the service systems, and workload burden (Berger and Mizrahi 2001; Bogo and McKnight 2006). In addition, upwards of half of all behavioral health services are using task-shifting program models that employ supervisors with clinical degrees to oversee staff without clinical degrees to provide the direct services (Hoge et al. 2007).

Many challenges to the provision of supervision have existed at multiple levels (e.g., policy, organizational, or individual) with increasing complexity combined with shrinking resources in the field (Hoge et al. 2011). Non-profit organizations, and increasingly co-located integrated health settings, have often been the providers of public mental health services. These organizations have been operating within a policy environment focused on measuring the outcomes (rather than the process) of the work with people seeking services (Mosley and Smith 2018). This focus on accountability via outcomes measurement has required organizations to expand their capacity to record their work with people seeking services and has ultimately increased the documentation requirements for providers.

States primarily fund community mental health services and the professionalization requirements for the workforce vary by state. Within a given state funding context, organizations may be directed to hire people with licensure and credentials and be required to provide supervision. Some states have also endeavored to bolster supervision practice by offering additional training and support for clinical supervisors as well as by specifying particular supervision guidelines (e.g., Connecticut Workforce Collaborative on Behavioral Health Supervision Competency Development Initiative; Supervision Competency Workforce Initiative 2009). Within these state policy contexts, organizations may offer robust clinical supervision that fulfills licensure requirements to attract a limited qualified workforce (e.g., in New York State one of the requirements for a clinical social work license is 3 years of supervised practice experience). In addition, particular programs (e.g., Assertive Community Treatment; New York State Office of Mental Health 2007) may have different requirements for supervision as part of the fidelity to the program model, which could influence supervisory practices in one program over another.

In response to shifting service contexts, the format of supervision has evolved (e.g., individual versus group and formal versus informal (or ad hoc) supervision time) (Schoenwald et al. 2009). In addition, supervision content is shifting. Supervisees do not always have access to supervision focused on clinical practice (Bearman et al. 2013; Ellis et al. 2013). Administrative supervision dominates limited supervision time and supervisors feel “ill-equipped” for additional responsibilities (Dill and Bogo 2009; Hoge et al. 2016). This is a critical issue of service quality that could result in potentially alarming clinical autonomy for an untrained workforce operating in an increasingly complex system of care (Choy-Brown et al. 2015; Substance Abuse and Mental Health Services Administration [SAMHSA] 2012, 2013). This administrative dominance may be attributed to states’ reliance more on requirements rather than financial incentives for supervision activities, as supervision has remained an unreimbursed requirement for service delivery. Such incentive structures may increase the focus on more administrative tasks (e.g., documentation for risk management and billing) and limit clinical practice supervision time. Supervisors carry the risk of vicarious (and sometimes direct) liability for any incompetent services (or omission of services) that constitute malpractice. Thus, supervisors are legally responsible for all activities of any staff member, volunteer, or intern they oversee (National Association of Social Work Insurance Trust [NASW] 2004; NASW & Association of Social Work Boards 2012). Given the potential impact of the organizational context on available supervision practice, it remains unknown how much discretion supervisors have to shape the supervision process.

Research has been mixed on access to adequate supervision. One national study reported that on average respondents working in behavioral health settings had access to effective supervision as defined by a 14-item measure of supervision effectiveness (Laschober et al. 2012). However, another study found that 93% of their sample of supervisees from a variety of community settings (i.e., 50% behavioral health, 16% University-based settings, 7% schools) had experienced inadequate supervision (Ellis et al. 2013). Yet, another study found that the majority of supervisors and supervisees in children’s community mental health in San Diego County were both satisfied with the amount of time spent on various functions in supervision and felt that evidence-based practice techniques were not included consistently enough (Accurso et al. 2011). In addition, the 2004 National Social Work Workforce study reported decreased supervision and increased workload (Whitaker et al. 2006). While some recommendations on how to bolster supervision to ensure safe and effective care have been made, questions remain about whether supervision is an available resource for clinical practice improvement in routine services (Dorsey et al. 2018; Hoge et al. 2014). Knowledge is limited about routine supervision practices, the use of best practice content, the quantity of supervision time devoted to clinical practice, and supervision models that are effective in integrating best practices into usual care (Dorsey et al. 2013). As such, this study seeks to contribute what supervision is available in routine practice within community mental health settings.

Methods

Data Source

This descriptive study examined data drawn from the baseline survey administered as part of a multi-state federally-funded randomized controlled trial testing the effectiveness of Person-Centered Care Planning (PCCP). Community mental health research sites (N = 14) were randomly selected in two northeastern states to participate in the trial. The community mental health sites collectively served over 8000 service users and provided a multitude of services including crisis intervention, individual and group therapy, residential, and case management services. Participating research sites nominated leaders, supervisors, and direct care staff to be recruited for participation in the parent study trial. In general, inclusion criteria primarily consisted of working for at least one year in a participating program; for leaders (N = 49), overseeing PCCP implementation; for supervisors (N = 81), overseeing service planning—the target service activity for the PCCP intervention; for direct service providers (N = 143), that they met the criteria of a change agent (e.g., leaders among staff) and were reporting to a participating supervisor. Each supervisor nominated two direct service providers.

Each of the participating providers received an email introducing them to the study from the leadership in the organization. Subsequently, surveys were administered via an electronic link embedded in an email from the research study team. The 45 min online survey was administered to research sites one month prior to the delivery of the PCCP intervention between October of 2014 and November of 2015. LimeSurvey software was used for survey development and data collection. The overall response rate for the survey was 89%. The human subjects committee from the authors’ institution approved all study protocols and the parent study was registered as a clinical trial. More information about the parent study methods has been published elsewhere (Stanhope et al. 2015).

Measures

This study examined provider reports of supervision (quantity and content) and contextual factors (research site, organizational role, and program type).

Supervision quantity was measured using two continuous items including the number of hours in supervision in an average week (“In an average week, how many hours do you spend in supervision?”) and the percentage of supervision time spent focused on clinical versus administrative tasks (On average, what percentage of supervision time is spent on administrative versus clinical practice content?). These two continuous items were used to calculate supervision quantity in which the percent of supervision time focused on clinical work was multiplied by the number of supervision hours. Three different surveys were administered based on the three possible participant roles (i.e., leader, supervisor, and direct care provider). The question stem texts varied, but all stems directed the participants to comment on the direct care supervision. Leaders were asked the amount of supervision time and format utilized for direct care providers in their agency. Supervisors were asked the amount of time that their direct supervisees were in supervision and the formats they utilized with their supervisees. Direct care providers were asked to speak about their experiences in supervision.

Supervision content was derived from one categorical item asking participants to endorse all applicable supervision formats or activities from a list of 11. For direct care providers, the question stem was: What formats for supervision do you receive? (e.g., individual, group, staff meeting, case presentations, live supervision, direct observations or recorded sessions, client feedback, peer supervision, consultant supervision, outside/independent, informal). Supervisors reported on the formats or activities available to their supervisees and leaders reported on the formats or activities received by direct care providers at their agency. A dichotomous variable was created from that item, with a “yes” answer meaning that the participant endorsed at least one of three possible best practice supervision activities from the list of 11 options itemized in the survey.

These three best practice supervision content activities were identified through a multistage integrative literature review (Whittemore and Knafl 2005) involving SAMHSA’s National Registry for Evidence-Based Programs and Practices (NREPP) to identify supervision characteristics associated with evidence-based practices and through a review of supervision literature for a broader perspective on best practice supervision activities. Inclusion for evidence-based practices in the review was determined using a four-stage process utilizing the NREPP search tool:

  1. 1.

    Evidence-based practices (EBP) were identified for mental health and substance use treatment of service users older than 13 years of age in outpatient or community settings.

  2. 2.

    A quality assessment review was performed of SAMHSA descriptions for Readiness for Dissemination and Quality of Evidence index scores of 3 or higher.

  3. 3.

    SAMHSA information was reviewed for the specification or emphasis on supervision practices.

  4. 4.

    A literature search was conducted to identify published work related to identified EBPs and supervision factors (Fig. 1 presents process flow diagram).

Fig. 1
figure 1

Multistage identification of supervision using SAMHSA’s National Registry of evidence-based programs and practices

Full size image

From this process, 12 EBPs were identified that utilized three different supervision models for the interventions: Motivational Interviewing, Cognitive-Behavioral Therapy, and Multi-Systemic Therapy (Henggeler and Schoenwald 1998; Martino et al. 2006; Milne 2009; Milne and Reiser 2017). Taking a modified common elements approach (Chorpita et al. 2007), these supervision manuals were reviewed for distilled components including the supervisory practice structure (i.e., quantity), content (e.g., activities), data source (e.g., taped sessions), characteristics, and available impact research. Next, these components were reviewed across the three manuals and shared elements were found across the supervision manuals, including opportunities for observation of actual practice and constructive feedback based on that practice (Choy-Brown 2016). This was found to be consistent with the literature in evidence-based supervision practices (Bearman et al. 2013; Milne and Dunkerley 2010) and best practice supervision (Borders et al. 2014). In particular, supervision content included live or direct observation, recording of EBP use with service users, or use of service user outcomes data to inform the session.

Other variables included were research site and organizational role, which were identified for each participant through administrative data prior to the survey administration and were not included as items in the survey. In the survey, participants provided information about demographics, caseload, and program type. Program type was measured by one categorical item asking respondents to select from the list of program types the one that best described their program (i.e., community support, Assertive Community Treatment (ACT), outpatient therapy, young adult services, residential services, other). Caseload was measured by one continuous item asking direct care providers to report on their own caseload and supervisors/leaders to report on the direct care providers’ caseloads on average.

Analytic Strategy

Univariate statistics identified the available supervision quantity and content in community mental health services. Independent samples t-tests, analyses of variance, and Chi square analyses identified statistically significant differences across research site (N = 14), program type (N = 6), organizational role (N = 3), and state (N = 2). This survey had limited missing data. Cases with missing data were deleted list-wise from analyses.

Results

Sample Characteristics

The sample (N = 273) was comprised of leaders (N = 49), supervisors (N = 81), and direct care staff (N = 143). The majority of participants were female (71.1%) and white (65.9%). The mean age of the sample was 42.6 years. On average, it was an experienced sample with an average of 13.89 years in mental health services and 7.53 years of tenure at their current organization. Approximately half of the participants held master’s degrees (55.3%), a third had bachelor’s degrees (33.8%), and others had less education. On average, supervisors oversaw 10.40 staff (SD = 9.05). Table 1 outlines sample demographics in more detail by role.

Table 1 Sample demographics by role
Full size table

Available Supervision Quantity

On average, participants reported 2.17 h for supervision quantity (i.e., supervision devoted to clinical content; SD = 1.93) with a modal number of 30 min and a median of 1.60 h. Please see Table 2 for descriptive statistics of supervision quantity and content by role and program type. Even if we assume that the clinical content encompassed primarily clinical topics (e.g., not educational content) and supervisors need the current status of each person served, this only gives supervisors approximately 5 min to learn about each person served with average caseloads, as reported by direct care staff, being 24.16 (SD = 22.83). On average, 58.9% of supervision was used for clinical-focused content while the additional 41.1% was used for administrative matters. Analyses of variance demonstrated significant differences in available clinical supervision time across research sites (p < 0.05) and program types (p < 0.05; see Table 2 for means and Table 3 for results). Independent samples t-tests identified significant differences between the mean supervision quantity in the two state contexts (p < 0.01): one state mean was 2.83 h (SD = 2.57) and the other was 1.90 (SD = 1.54). The range of mean clinical hours for organizations across both states was 1.40 (SD = 1.08) to 3.39 (SD = 2.45).

Table 2 Results of Chi-square tests and descriptive statistics for supervision by role and program type
Full size table
Table 3 One-Way Analyses of Variance Analyses of Supervision Quantity by Role, Program, and Site
Full size table

Available Supervision Content

As defined, supervision content was minimally available with an average of only 18.2% of direct care staff and 28.6% of all participants endorsing access to at least one of the best practice supervision content activities. Chi square analyses revealed no significant differences in supervision content across states (χ2(1) = 0.297, p = 0.586), research sites (χ2 (13) = 16.62, p = 0.217), or program types (χ2(5) = 4.40, p = 0.494). However, significant differences in supervision content were found across identified role in the organization (χ2 (2) = 0.16.81, p < 0.001). Table 2 presents descriptive findings for supervision content by program type and role.

Discussion

These study findings suggest that providers have access to an adequate dose of supervision quantity with an average of over two hours per week focused on clinical content. This time includes all potential modalities for contact (e.g., staff meetings, individual, and informal supervision time). In addition, respondents indicated that they spent an additional 1.5 h per week on average in supervision focused on administrative content. These findings differ in total quantity and percentage of time devoted to clinical content when compared to previously reported findings of about 50 min every other week (Dorsey et al. 2018). Findings here suggest an adequate amount of supervision time as compared to NYS clinical licensing standards (New York State Education Department 2017) or CT requirements for supervisory contact that require 1 h per 40 h of clinical contact (Supervision Competency Workforce Initiative 2009). However, caseload burden may have limited supervisors’ ability to talk with providers about their interactions with people seeking services each week. The average time constraint calculated in this study indicates a potential challenge to supervisors’ critical function of overseeing service interactions.

Findings suggest that while the quantity of supervision met recommended standards, supervision content consistent with best practice supervision standards or supervision activities utilized in efficacy trials of evidence-based practice were not widely available in routine services in these community mental health settings. Best practice supervision content was available for a limited number of participants in this sample, with under one-fifth of providers reporting access. This finding is lower than previous findings that at least one-third of providers report observation or review of their work informing supervisory feedback (Laschober et al. 2012). This opportunity for constructive feedback based on review of providers’ practice—whether from the service user or supervisor perspective—was a consistent element in specified supervision models used in clinical trials. Feedback has been considered an essential part of successful learning (Milne 2009). For a workforce primarily relying upon on-the-job learning (SAMHSA 2013), this could signal a key missed opportunity and a clear contextual barrier to integrating new practices into available mental health services. Kolb’s (1984) Experiential Learning Theory has proposed that skills and knowledge are acquired through a process of practical experience, reflection, conceptualization, and planning. To inform this learning process and meet service user needs, supervisors and providers have relied on provider self-report and progress notes (Accurso et al. 2011). The discussion of provider self-report and progress notes have potential strengths for understanding the clinical encounter, but also introduce potential obfuscation of key information given the nature of the supervisory relationship (Noelle 2003). Opportunities for external practice review and feedback may provide insights into areas for improvement and share the substantial responsibility for practice quality with others. Additionally, supervisors make decisions about how to use their time in supervision and address the potential tensions between the records of practice and the actual practice quality. Supervisory decision-making about what and how much supervision they provide may also be sensitive to contextual factors including considerations of potential liability.

This study also examined how supervision quantity and content varied as a function of the state, research site, program, and role that participants occupied within their organizations. For supervision content it largely depended on who was asked, with leaders being significantly more likely to endorse the availability of best practice supervision activities. This significant difference in supervision content, but not in quantity could have indicated a disconnect between leadership perceptions and on-the-ground realities of service provision. Leaders may have a general sense of how much, but not what or how, supervision takes place in their organizations. Alternatively, leadership responses could have also represented awareness of at least one best practice supervision activity available within the programs as opposed to availability throughout the organization. Beyond these role differences, supervision content did not vary significantly across the states, research sites, or program types within this sample, potentially indicating a consistent unavailability of feedback based on observation or review of practice.

On the other hand, supervision quantity was found to vary by contextual factors including state, research site, and program type. States typically administer a significant portion of the community mental health services funding and are in a position to influence how supervisors’ time is used within their system. However, what remains unclear are the mechanisms of state influence on supervision and how they monitor or incentivize supervision quantity and content. The variation among participating sites suggests that organizations are also making choices about supervision quantity, but have less often prioritized the use of best practice supervision content during that time. Programmatic variation in supervision quantity seems to mirror program enrollment criteria for severity of need. For example, outpatient programs received the least supervision whereas ACT teams received the most. Alternatively, supervision hours could have perhaps varied by program according to the amount of staff’s clinical training (e.g., outpatient therapy providers presumably have more clinical training and therefore receive less supervision).

These variations support that supervision quantity can be malleable with regards to shifting contextual factors and that key stakeholders make choices about supervision dose and content. The range among direct care providers indicates variation between and within supervisors themselves in supervision quantity and content. Such findings suggest that supervisors also exercised discretion in how much and how they used their time with their staff. Another study found that staff perceived supervision time alternately as either support or scrutiny of their work, which was found to be influenced by the process and content during supervision time (Choy-Brown et al. 2015). Further uncovering the potential implications of supervisory choices in supervision represents an important opportunity to understand the role of supervisors in shaping direct practice.

In the administration of community mental health services, supervisors have considerable responsibility and discretion over the service user experience of services. In environments that are over-saturated with work, policy makers and organizations often cannot realistically expect all of the stated policy requirements to be met (Lipsky 2010). In fact, bureaucratic effectiveness has often relied on managers and supervisors to calibrate policy to the on-the-ground landscape (Evans 2011; Lipsky 2010). In particular, during an implementation effort to translate a new practice into their work, the supervisors’ understanding of that practice and their motivation in facilitating that understanding among their staff will potentially affect how they use their discretion in influencing practice. Given the stress and overburdened work levels, supervisors in these bureaucracies cannot meet all requirements and thus must prioritize their time and energy (Lipsky 2010). Should a conflict arise between the supervisory and organizational or external policy environments, the supervisor will ultimately be in a key proximal position to facilitate or inhibit the new change and to make sense of the critical practice elements within the intervention. According to previous research (Rapp et al. 2010), this has perhaps been particularly true for the translation of complex interventions with underlying value shifts (e.g., recovery-oriented practices) in addition to concrete practice changes (Whitley et al. 2009). This study supports the notion that contextual factors contribute to the availability of best practice supervision, due to finding a difference of over 2 h per week in supervision quantity between some sites and a range of 15–55% of staff endorsing access to best practice supervision content. However, mechanisms to improve that availability remain underdeveloped. Research and practice knowledge suggest that supervisors affect practice quality and that a better understanding of supervisory mechanisms (e.g., supervision quantity and content) within their supervision time would further bridge the gap in the relationship between supervision and practice quality improvement (Milne 2009).

Limitations

This study has contributed new knowledge of the availability of supervision quantity and content. However, findings must be understood in the context of the limitations of the study. In particular, the evaluation of statistically significant differences among variables has not considered the complex picture inherent in the real world context of these organizations. Community mental health services are incredibly diverse and complex, operating in equally diverse and complex systems of care. Future research employing multivariate analyses would strengthen our understanding of these relationships. These findings also represented participant self-report of their experiences with supervision, which are subject to potential response bias. In addition, these study data did not include alternative potential activities that states, organizations, and programs were engaged in to support the quality of their practice such as the implementation of other related interventions (e.g., motivational interviewing). Lastly, these findings support the need for further analyses examining all of these factors together to understand the contextual fabric of influences on supervision - why and how it matters, and how we can better maximize the opportunities available in this already embedded resource.

Conclusions and Implications

These study findings about the supervision quantity and content (as defined in this study) can contribute to future research, social work education, practice, and policy. Future research investigating supervisors and supervision should consider the influence of contextual factors within the system, organization, and program. In particular, the simultaneous consideration of the context, supervision, provider practice, and service user experiences using multivariate analytic strategies would further our understanding of the relationships among these variables. Given the paucity of best practice supervision literature, further research is urgently needed to better understand how organizations and providers approach practice quality support in routine care.

Additionally, for social work education, social work supervisors have played significant roles as field instructors supporting our “signature pedagogy” in field learning. While field education differs from practice supervision in key ways (e.g., purpose), these findings shed insight into available supervision quantity and best practice supervision content within these field settings. The integrative review of known best practice supervision indicated that the provision of feedback based on actual review of provider-service user interactions has been a critical component to improving and sustaining quality practice. However, this study’s sample did not report consistently receiving feedback. Practice quality may improve if supervisors and practitioners focus on the content over the quantity of supervision time. For practitioners, students, and supervisors, the consideration is how one might access and integrate opportunities for feedback into their supervision to support practice.

Beyond individual ethical responsibilities, these findings have implications for policy makers, organizational leaders, administrators, and educators. In addition, research has been increasingly examining the potential value of simulated practice interactions for learning and feedback (Bearman et al. 2017; Bogo et al. 2014). Further knowledge of simulation applications in low-resource settings is needed. Funders of community mental health services have historically prioritized their spending to be devoted to interactions with service users (e.g., fee for service billing). Tension continues to exist between supporting quality versus breadth of service provision within increasingly limited budgets and waning political support. The integration of best practice supervision that is known to improve supervisee learning and practice quality may require increased staff time and organizational infrastructure for data collection. Without resolution, unmet learning needs could continue to frustrate workforce training and retention and shift the responsibility for practice quality downwards from policy makers to organizations and, ultimately, to supervisors and individual providers. Financial difficulties may exacerbate these workforce issues, constrain organizational quality assurance activities, and limit opportunities for frontline staff to receive best practice supervision content. In turn, this could then contribute to poor quality of care, which has had considerable consequences for people seeking services in public mental health settings (Institute of Medicine 2015). Further examination and introduction of incentives for integrating best practices into clinical supervision, which is an already embedded resource for workforce training, could be critical for addressing gaps in the quality of care.