Introduction

Concurrently addressing the substance abuse and mental health issues of offender populations has been identified as a foundational element for successful rehabilitation (Lurigio, Rollins, & Fallon, 2004; Miller & Miller, 2011; MacKenzie & Hickman, 2006; Miller & Miller, 2010; Osher, 2007). Overall, the literature on what works in offender reentry specifies that holistic services individualized according to needs and situations that begin prior to release and maintained as aftercare are most apt to be successful (Lowenkamp, Latessa, & Hollsinger, 2006; Pearson & Lipton, 1999; Petersilia, 2004; Osher, 2006; Travis, 2005). Accordingly, the behavioral health court (BHC) concept has emerged as a promising treatment strategy. A problem-solving court serving offenders diagnosed with mental health disorders and, more often than not, substance misuse, BHCs synthesize close case management, cognitive change therapies, medicated assisted treatment, and crises intervention resources under judicial management.

The transition of offender programming from prisons to community corrections has fostered broad scale introduction of behavioral health and similar specialty and problem-solving court initiatives (Griffin, Steadman, & Petrila, 2014; Watson, Hanrahan, Luchins, & Lurigio, 2014; McNiel & Binder, 2007). Spearheading the offender reentry movement, most are funded through US BJA Second Chance or Justice Mental Health funding streams and include an evaluation component. Despite the requisite nature of first establishing program fidelity to more rigorously attribute programming outcomes to interventions, criminal justice program evaluation largely remains limited to the statistical analysis of relationships between programming activity and outcomes. Indeed, very few process evaluations are found within the criminological and criminal justice literature, reflecting scholarly lag behind momentum toward normalizing fidelity-focused research provided by prioritization language in some US Office of Justice Programs sub-agency grant announcements and ranking weightings in national evidence based programs and practices registries. In the National Institute of Justice sponsored evidence-based registry, crimesolutions.gov, for example, demonstration of program fidelity is requisite for a program or practice to be designated in the top category of “effective.”

The current paper features a fidelity-focused implementation and process evaluation, the first phase of a larger mixed methods evaluation of a jointly funded US BJA and SAMSHA BHC program delivered by the Louisiana 22nd Judicial District Court. After brief description of BHCs, generally, and the BHC evaluated, specifically, we relate the methodological approach utilized to ascertain implementation intensity and program fidelity through program mid-point. Findings relate the BHC’s performance to date, provide recommendations to the Court, and inform related fidelity research.

Behavioral Health Courts

BHCs, also commonly referred to as mental health courts, are designed to address the needs and minimize the public disorder and safety risks posed by people with mental health (MH) disorders (Kennedy, 2012). More often than not, MH individuals also suffer from substance abuse as indicated by the over-representation of dually diagnosed offenders in the system (Munetz & Griffin, 2006). Addressing the special needs of this large population is a multi-faceted challenge that must be met because, without treatment, recidivism is highly probable. If successful, BHC programming presents a win-win situation for all stakeholders. Offenders receive drug treatment and psychological services, individual and group therapy, and individualized post-release success plans centered on treatment continuation. To the extent that rehabilitation disrupts drug-crime trajectories, public safety is enhanced while offering potential system savings. BHCs vary considerably across jurisdictions in terms of targeted offender populations and treatment modalities but most feature common elements such as a specialized court docket, individualized community-based treatment plans, regular court hearings for status review, and disciplinary action for non-compliance per judicial discretion informed by a collaborative team comprised of community corrections and treatment professionals.

The Louisiana 22nd Judicial District BHC Program

The 22nd Judicial District Court of Louisiana, serving St. Tammany and Washington Parishes just north of Metropolitan New Orleans, started its BHC program in November of 2011. As a new edition to an existing suite of problem solving courts in the jurisdiction, the program was designed to align with the National Drug Court Professionals Association specified ten essential elements of drug courts, modified for a mental health population. The BHC operates as a post-conviction alternative to traditional sentencing for probation-eligible misdemeanor and felony offenders with debilitating mental illness. Validated screening tools, the Mental Health Screening Form III (MHSF-III) and the Texas Christian University Drug Screen-II (TCUDS-II), were utilized to assist case managers in determining eligibility based on severity of mental illness with particular focus on diagnosis of a clinical disorder (as defined under Axis I disorders in the DSM-IV-TR) or a personality disorder (per former Axis II diagnosis).

A few months after inception, the BHC suffered from considerable setbacks in service capacity primarily due to a statewide switch to privatized mental health services and the closure of local state-run behavioral health clinics and hospitals. These shifts proved particularly problematic within the Parishes as the jurisdiction is within the lowest 20th percentile in the nation for per capita public funding earmarked for mental health services. More specifically, the jurisdiction is located within the bounds of the Florida Parishes Human Services Authority (FPHSA) which is a local governance entity created to pool funding dollars for addictive disorders, mental health services, and developmental disabilities. Unfortunately, the Louisiana Legislature gives different per capita funding to each locality and, despite pronounced need, the FPHSA gets the least per capita funding in the state. In comparison, the Metropolitan Human Services District, which includes Orleans, St. Bernard, and Plaquemines Parishes, receives more than double in per capita funding. Taking into consideration that Louisiana ranks 43rd out of 50 for per capita mental health expenditures, the 22nd JDC is located amongst the “lowest of the low” areas nationally for mental health care funding.

Such generalized disadvantage presents a severe challenge for those with mental illness and particularly for those with co-occurring disorders. The stakeholders in the jurisdiction had identified a persistent services gap for offenders exhibiting mental illness symptomology early on in justice processing. Individuals in need of treatment were facing significant delays in beginning therapeutic regimens that, in turn, often worsened release options for this vulnerable population. Through the BHC, however, the jurisdiction has been able to be innovative with services delivery so as to provide proactive and responsive care.

The BHC program is staffed by a team of various professionals, including a supervising judge, public defender, district attorney, case managers, treatment providers, program director, and medical officers. These partners work together to assist BHC clients in obtaining access to treatment, housing, and transportation and are inundated with troubleshooting client issues that, if unabated, could spiral into broader substance abuse and MH problems and potential public safety risks. Accordingly, the BHC features Illness Management and Recovery (IMR) as the primary intervention modality (SAMHSA, 2010; Mueser et al., 2002; Mueser, Bond, Drake, & Resnick, 1998) which is augmented by the quick response, environmental intervention Assertive Community Treatment (ACT) to address crisis situations (Morrissey, 2013) as described below.

Logistically, the BHC is an 18-month multi-phasic program (three 6 month phases). In Phase I, clients receive weekly treatment, are drug tested at least twice a week, and are required to attend three 12-step meetings per week, at least one of which is particularly designated for individuals with co-occurring disorders. Case management includes daily check-in with clients either in person or by phone, maintenance of a medication journal documenting dosing compliance, and proof of attendance at treatment appointments. Finally, clients attend weekly status meetings with the district judge overseeing the BHC. Upon compliance with these requirements and three “clean” months, they advance to Phase II per treatment team discretion.

Phase II replicates Phase I with slightly lowered compliance requirements: bi-monthly status hearings (instead of weekly), program participants must be actively seeking employment, meet community service requirements, and obtain a GED or cooperate with Louisiana Rehabilitation Services for vocational assessment and training. Advancement to Phase III is achieved after full compliance with Phase II activities and presenting clean urine for at least an estimated three additional months. In Phase III, status hearings are conducted monthly, clients report on an “as directed” basis, drug testing is reduced to once a week, and individual counseling is set for two contact hours per week, but adjusted per need. Program graduates are referred to a year of aftercare with no court attendance expectations, but continued check-ins with a case manager and probation officer are required. Aftercare includes referral and connectivity with community coalition partners, faith-based organizations, and social services to help BHC program graduates continue self-sufficient and healthy lifestyles. Last, random drug testing and treatment continues as dictated by individualized treatment plans, but a 6 months clean period and full compliance with all phases are required for program completion.

With regard to the primary treatment strategy, IMR was selected because of its proven effectiveness in addressing co-occurring illness (Mueser et al., 2002). IMR facilitates desired mental health outcomes such as decreased hospitalization and substance abuse, but also seeks to reduce the arrest rate and time spent in jail by addressing criminal behaviors in treatment. Osher and Steadman (2007) contend that IMR is easily adapted to address the unique aspects of justice-involved individuals based on the common application of psycho-education and cognitive-behavioral approaches to offender rehabilitation.

The overall BHC strategy features a subset of evidence-based practices with proven effectiveness for justice-involved individuals with a diagnosed mental illness or co-occurring disorder, including: Motivational Interviewing (MI), Relapse Prevention Therapy (RPT), Cognitive Behavioral Therapy (CBT), and Intensive Care Management (ICM). Each modality’s effectiveness for the target population is well-documented in the treatment literature; for example, MI is applicable in any situation in which engagement in therapy and the process of change is needed/desired. Of particular interest to the BHC program, MI has proven effective in the treatment of anxiety (Westra & Dozois, 2008), as a prelude to psychotherapy of depression (Zuckoff, Swartz, & Grote, 2008), for co-occurring disorders (Martino & Moyers, 2008), and for medicated assisted treatment compliance (McCracken & Corrigan, 2008). Additionally, MI has been applied in criminal justice settings to promote engagement in and reduce resistance to treatment (Ginsburg, Mann, Rotgers, & Weekes, 2002). IMR is featured in the SAMHSA Evidence-Based Practices KITs series (SAMHSA, 2010) and MI is listed in the SAMHSA National Registry of Evidence-Based Programs and Practices for substance abuse.

While IMR is the primary intervention, Assertive Community Treatment (ACT) is a proven evidence-based treatment protocol wherein a team of mental health and other professionals (e.g., psychiatrist, counselors, substance abuse specialists, vocational rehabilitation specialist, and case manager/team leaders) target services delivery to a small caseload of individuals with severe mental illness in community settings (Morrissey, 2013). Treatment services are out-sourced to clients instead of reliance on voluntary participation, available clinic space, or transportation. This means that, for people who have issues obtaining transportation or may be too sick to comply with even simple participation directives, ACT team members can work around barriers to ensure treatment. Supplementing IMR and ACT, all BHC clients receive intensive case management, judicial monitoring, program compliance monitoring via random drug testing, vocational rehabilitation, transportation, and housing assistance as needed.

Evaluation Strategy

Evaluation of the Louisiana 22nd District Court’s BHC intervention is a rare opportunity to execute a rigorous mixed-methodological “gold standard” research design comprised of: 1) a process evaluation relying on qualitative methods to assess implementation intensity and program fidelity and 2) an experiment featuring the random assignment of participants to experimental and control conditions to statistically isolate the effectiveness of the intervention. Of relevance here, the study was designed to answer three process-specific research questions regarding BHC program implementation and process: 1) Does the program adhere to evidence based practices with documented success in addressing mental health disorders and substance abuse within community correctional settings?, 2) Does the program deliver treatment in a manner consistent with prescribed program protocols thereby demonstrating program fidelity?, and 3) Are services concentrated during the high risk period of transition from incarceration to the community?

Mixed methods research strategies offer advantages over singular methodological designs and first entail qualitative steps to establish that treatment services as delivered adhere to the selected modality’s theoretical design in terms of dosage, length, and intensity (Miller & Miller, 2015, 2016). While qualitative findings contextualize statistical findings – the diminishing yet most commonly cited purpose of qualitative evaluation in criminology and criminal justice, the primary function of qualitative program evaluation is fidelity confirmation so as to further reduce spuriousness by elimination of program adaptation as a rival outcome driver (Emshoff et al., 1987; Esbensen, Matsuda, Taylor, & Peterson, 2011; Copes & Miller, 2015). The import of establishing fidelity is evidenced, in part, by the unprecedented emphasis on accountability reflected in both the aforementioned federal justice system funding announcements and national evidence based program registries’ rating criteria.

Evaluation of the BHC was administered through the University of North Florida Institutional Review Board with approved human subjects protocol (i.e., informed consent forms) executed prior to the onset of research activity. Data collection focused on program content and its delivery to assess adherence to BHC program content/protocol, quantity and duration of services, quality of delivery, and participant responsiveness. We also sought to identify and contextualize possible problems, barriers, or other issues relevant to program operation and success as reflected across major program domains. The research design entailed six site visits over a 3 year perod, half of which generated the data presented here a year after initial program implementation.

Research techniques included direct observation of court appearances by the majority of program participants, in-depth interviews with the 22nd Court’s administration and program staff including referring Judge Peter Garcia and former Court Administrator Adrienne Stroble, and members of their staff, ACT service providers, and, perhaps most importantly, focus group interviews with BHC program participants (see Table 1 for an overview of participant characteristics). Additionally, we observed multiple treatment sessions and three monthly case review meetings wherein progress reports and discussions of each client transpired.

Table 1 BHC program participant characteristics (N = 26)
Full size table

Primarily, we sought to empirically substantiate program integrity in terms of modalities employed, adherence to the principles of effective intervention, and grant conditions compliance (Lowenkamp, Latessa, & Smith, 2006). Interviewing offenders receiving services enabled consideration of their daily treatment experiences, in this case the nature of IMR and ACT services from the participant’s perspective. Participant input has proven useful to identify environmental features perceived as barriers to recovery (Miller, Koons-Witt, & Ventura, 2004) and whether programming is effecting participant engagement (Miller, Tillyer, & Miller, 2012).

Site visits included two 90 minute focus group interview sessions per trip with active program participants (5–8 per group). In-depth and focus group interviews were guided by semi-structured questionnaires to ensure systematic topic coverage and data collection across groups and individual respondents and the Justice Program Fidelity Scale (JPFS), a site visit based inter-rated fidelity instrument (see Table 2), that enabled measurement of fidelity across program domains. Domain indicators can be weighted as informed by theoretical emphasis on specific modalities or treatment elements. By establishing the BHC’s degree of fidelity, subsequent statistical findings regarding program impact on relapse and recidivism can be observed with greater confidence that results are a function of IMR and ACT programming rather than a modification of the treatment plan, other intervening variables, or coincidence.

Table 2 Justice Program Fidelity Scale
Full size table

Findings

The question of whether the BHC program design incorporated evidence based treatment elements was satisfied per specification of the various modalities described above. After establishing that the program was implemented according to an evidence based design and that services were launched and sustained with integrity and as specified in the treatment plan, the research team turned attention to programming fidelity and the focal concern of whether program adaptation (variance) had occurred and, if so, to what extent (Blakely et al., 1987; Grote, Swartz, & Zuckoff, 2008; Lau, 2006). To assess treatment plan delivery compliance, we rated the various treatment elements comprising the holistic orientation of the Louisiana 22nd District’s Behavioral Health Court as presented in Table 2.

The five program domains measured on the JPFS specify the BHC’s implementation intensity and modality compliance, four of which were rated for this implementation and process evaluation phase. Adherence indicates whether treatment implementation and delivery is consistent with program design. Review of intake screening instruments, treatment plan components including individualized services and dosage schedules, presentation of intake appointments, assessments, and caseload compliant with grant expectations collectively demonstrated program adherence to the intervention design. Exposure was indicated by participant contact hours in counseling sessions and other services, number of sessions delivered, and duration of sessions and the program, overall, which was confirmed as consistent with the treatment strategy per cross-referenced data from various stakeholders.

Delivery quality refers to the level of treatment as indicated by staff credentials, attitude, and continued training participation and participant engagement refers to the extent of client “buy-in” to treatment objectives as indicated by attitude and degree of involvement. Our high rating of delivery quality was particularly influenced by two notable program features. First, the program is comprehensive with client needs and supervision balanced and synthesized for program success as indicated by the number and various elements of professional services represented at regular BHC client case review meetings, which also further intensifies and evidences individualized treatment. Also, judicial involvement is typically limited to referral and then court interaction rather than the hands-on oversight demonstrated by Judge Peter Garcia. His organic comments during case review discussions of participants’ circumstances and needs suggests involvement and familiarity with participant progress, specifically, and a substantial time commitment to the BHC, generally. This level of participation in the alignment of program services is much more pronounced than with the majority of specialty court programs we have reviewed (e.g., Miller, 2014; Miller et al., 2012; Miller & Miller, 2010, 2011) The last domain, program differentiation, refers to whether services are delivered consistently over time and if program size, individual counselor caseloads, and dosage remain approximate across cohorts – a temporal reality difficult to confirm at program midpoint.

As a site visit based and inter-rater reliability driven instrument designed for justice program evaluation, the JPFS was used to capture BHC performance to date. In general, there appeared to be little adaptation and fairly strict compliance across treatment programs and conceptual domains, indicating that: 1) the BHC was implemented with suitable intensity and integrity to be reflective of the employed modalities (IMR and ACT) and 2) data from multiple stakeholders across three site visits demonstrated approximate consistency in regards to time in treatment, caseloads, participation requirements, and the regular delivery of treatment components, as well as confirming stakeholders’ attitudes regarding the treatment climate and experiential commonalities. Attendance at the case review and triage meetings attended by the entire BHC staff allowed us to hear detailed descriptions of offenders’ progress and plans to redirect resources to troubleshoot crisis and noncompliance situations, satisfying that resources were being concentrated during critical junctures in the treatment process.

Recommendations

While observations during implementation often reflect start-up difficulties, the BHC program serving St. Tammany and Washington Parishes was launched with integrity and clearly features evidence based design and a strong professional orientation to services delivery. Programming, generally, can benefit from minor program adjustments toward intensifying impact. We are mindful that additional data collected through the remaining site visits will enable a determination more reflective of the program’s overall fidelity and quality of services delivery. While we will continue to gauge services delivery until the end of the evaluation period, our initial ratings suggest high levels of program fidelity across all five assessed domains. In two domains, however, we observed activity less than optimal as discussed below in the form of recommendations and improvement opportunities for enhanced BHC performance.

Observation of group treatment sessions, for example, seemed to lack an evidence-based orientation generally deemed necessary for optimal success. Specifically, there were no discernable treatment protocol, curricula, or agenda in place to structure therapeutic sessions for treatment-as-usual participants. An evidence based approach in observed services delivery was simply not apparent as the session seemed to be little more than a round-robin conversation wherein participants related, typically, “things are going okay” and any life news such as obtaining employment or ending a period of some sanctioning in the program. More troubling, the counselor could not articulate the primary nor any other modalities being employed. Further effort to document evidence based substance in theses sessions was made through our varied queries of what indeed was being delivered; unfortunately again, only vague therapeutic jargoned responses around a general theme of “social support” were provided leading to the above conclusion regarding this supplemental program component.

In regard to this or any other post-implementation program component at this juncture of mid-point assessment, we are hesitant to conclude that the therapeutic sessions are counter-productive; it could be that the counselor is unable to articulate objectives – a potential problem given that counseling success ostensibly derives from verbal communication effectiveness. Additional site visits should indicate either that our assessment to date is a function of observation on atypical days or that this program element needs address and intensification so as to be of arguable value to BHC objectives rather than its present largely demonstrative function. The infusion of more structure is likely needed both in general and to ensure minimal treatment variation across clinicians as some extent of judicial and/or treatment staff turn-over in a program the size of this BHC initiative is very likely over the life of the grant.

Participants receiving Assertive Community Treatment appear to be receiving concentrated services per multiple narratives during observations of court appearances, staff meetings, and focus group interviews. A major impending process phase evaluation step is to interview ACT providers and observe ACT treatment activity, not reportable here pending a December 2015 scheduled site visit. Program staff have self-identified the need for additional resources in treatment capacity, including providing for additional training in trauma-informed care options - thus satisfying the JPFS sub-item measure of staff qualifications. Open communication with treatment providers clearly informs the ACT team of gaps in care and enables feedback regarding needs; it is unclear whether a formal process is in place to ensure feedback on a frequent basis, particularly as gaps in care are being identified, so that IMR is aligned optimally with ACT toward celerity of intervention.

The qualitative data collection techniques presented here can be coupled with interview and fidelity instruments in modified combinations to address a wide range of justice system programming. The JPFS is highly customizable in terms of both domain measurement and component flexibility. In that specific modalities emphasize different treatment strategies and vary in regard to the logistics of treatment regimes, the domain indicators may be weighted for emphasis as indicated by theory. The fidelity scale is suitable for implementation and initial process evaluation through assessment of the first four domains and final fidelity determinations through inclusion of assessment of program differentiation that is better determined over time. Primarily though, the JPFS identifies conceptual domains by which to holistically appreciate program definition and dynamics. Assessing programming holisitically by incorporating a process oriented phase is simply more scientifically rigorous than continuing purely quantitative evaluations that may erroneously assume program integrity and attribute outcomes to treatment that reflects modalities and practices in name only.