The Criminal Justice Drug Abuse Treatment Studies (CJ-DATS) is a multisite research cooperative funded by the National Institute on Drug Abuse, a component of the National Institutes of Health (Ducharme et al. 2013; Fletcher and Wexler 2005). The cooperative brings together academic and research institutions, each partnered with one or more criminal justice agencies and community treatment providers, to engage in implementation research projects designed to address identified gaps in the service delivery system for adult offender populations.

Health service delivery, particularly for correctional clients, often requires coordination between distinct organizational entities. Many individuals with mental health needs or substance use disorders are under the supervision of community corrections. Treatment services, however, particularly medication-assisted treatment (MAT) for opioid and alcohol dependence, are rarely delivered by correctional staff (Friedmann et al. 2012, 2013). Rather, receipt of treatment services typically requires that individuals be referred to treatment providers in the community. Such referrals to care are dependent, however, on the functioning of inter-organizational relationships (IOR) between community corrections and treatment organizations. This paper describes a structured effort to improve IORs between community corrections and treatment providers in order to increase access to MAT for offenders under community correctional supervision, by testing an implementation intervention in a multisite cluster randomized trial (MSCRT).

The study, MAT Implementation in Community Correctional Environments (MATICCE), focuses on treatment options for drug-involved offenders that include referral to MAT, that is, any treatment for a substance use disorder that includes a pharmacologic intervention as part of a comprehensive substance abuse treatment plan with an ultimate goal of patient recovery with full social function (Substance Abuse and Mental Health Services Administration (SAMHSA) 2014). In the US, MAT has been demonstrated to be effective in the treatment of alcohol dependence with Food and Drug Administration (FDA) approved drugs such as disulfiram, naltrexone and acamprosate (Johnson 2008); and in opioid dependence with methadone, naltrexone and buprenorphine (Amato et al. 2005). A National Institute of Health expert panel has recommended that opioid agonist treatment be made more widely available to both criminal justice and non-criminal justice populations (NIH Consensus Development Panel 1998).

Unlike effectiveness trials to develop novel treatments, or implementation studies which seek to promote the adoption of a particular clinical practice by an inexperienced group of providers, this study tested implementation strategies for linking correctional agencies and offenders with evidence-based treatment services that already existed in their communities. Although MAT was not universally available, each of the major metropolitan areas represented in this cooperative had at least one clinical setting in which those services were provided. Rather than approach adoption of MAT as a problem best solved by asking correctional agencies to expand their scope of responsibility and expertise, MATICCE began from the assumption that addressing two persistent barriers to service receipt—staff unfamiliarity with MAT and a lack of inter-organizational linkages—should result in more frequent and sustainable coordination of services for offenders transitioning between the correctional system and community supervision.

Theoretical Perspectives on Inter-Organizational Relationships

IOR among human service agencies can vary along a continuum ranging from the ad hoc, market-based delivery of services by local providers to the complete control and coordination of a fully integrated, centralized service delivery system. Many theorists (D’Aunno 1997; Friedmann et al. 2000; Oliver 1990; Samet et al. 1996) argue that stronger linkages can lower the barriers (“friction”) and increase access to service delivery. While some interagency linkages consist of formally structured arrangements for coordination, a far greater amount occurs in the form of short-term, ad hoc efforts (Van de Ven and Ferry 1980; Van de Ven and Walker 1984). These unstructured relationships are important because of their pervasiveness and their influence on client outcomes.

Three major dimensions are critical for human service agencies (Van de Ven and Ferry 1980). First, the perceived need for resources to achieve organizational goals is usually the most important factor that stimulates inter-organizational coordination. Resource dependence influences the development of inter-organizational communications and consensus, monitoring, monetary transactions, and client referrals. Second, the pattern of inter-organizational coordination depends on the types of resources being coordinated. Interagency relationships based on monetary transactions tend to have an impersonal and formalized mode of coordination, while client referrals may reflect a more personal and informal coordination pattern. Third, the complex role of consensus or conflict between agencies in the development of coordination is important. Early in interagency relationships, coordination develops in response to a perceived need for resources. This dependence, in turn, stimulates greater frequency of communications. However, as the parties begin to negotiate more specific methods for conducting transactions, inconsistencies in relational assumptions may emerge. This latent conflict can facilitate a drive for greater autonomy, i.e., less resource dependence, over time. Ven de Ven and Ferry emphasize the need for multidimensional and longitudinal perspectives to assess key changes in interagency relationships over time, especially in response to a newly perceived need for coordination.

Applying Implementation Science to Inter-Organizational Relationships

The emerging literature on innovation implementation initially had a heavy emphasis on observational studies that documented the natural diffusion of evidence-based practices under natural conditions (Friedmann et al. 2007; Knudsen et al. 2011; Lundgren et al. 2012; Miller et al. 2006). More recently, the field of implementation science has been moving towards studies that explicitly compare how implementation strategies may be leveraged to increase the attainment of specific objectives. As noted in a review by Powell et al. (2012), an implementation strategy is “a systematic intervention process to adopt and integrate evidence-based health innovations into usual care” (p. 124).

Major conceptual models of implementation (e.g., Aarons et al. 2011; Damschroder et al. 2009; Damschroder and Hagedorn 2011; Feldstein and Glasgow 2008; Greenhalgh et al. 2004; Proctor et al. 2009) posit that interactions between multiple agencies serving clients with mental health or substance use disorders can affect implementation practices, service delivery, and outcomes in numerous ways. For example, the Center for Mental Health Services of the U.S. Department of Health and Human Services developed the Access to Community Care and Effective Services and Supports (ACCESS) project to strengthen the interagency integration of community-level service delivery systems to improve the receipt of needed services by homeless persons with serious mental illness (Cocozza et al. 2000; Morrissey et al. 2002; Randolph et al. 1997). Integration of services was viewed as a strategy for meeting the multiple needs of persons who seek services in a fragmented system (Randolph et al. 2002). Improved integration was to be achieved, in part, via an intervention targeting selected local provider partners in 18 different communities (Morrissey et al. 2002).

Several planned activities in ACCESS were aimed at reducing system-level fragmentation. Strategies included building interagency coalitions, interagency teams for service delivery, cross-training, and/or developing interagency agreements or memoranda of understanding (Randolph et al. 1997, 2002). At the conclusion of the project, a higher level of integration was found at the experimental sites than the control sites (Morrissey et al. 2002). Contrary to expectations, however, better systems integration did not always result in improved client-level outcomes. While sites that had greater improvement in system integration were more likely to achieve stable housing for the homeless, researchers found no evidence that technical support and the allocation of funds for systems integration improved client outcomes such as referrals (Rosenheck et al. 2002). Overall, the ACCESS project suggested that implementation of systems integration strategies can be effective, but requires time, commitment and resources (Goldman et al. 2002; Rosenheck et al. 2002).

While these projects have made substantive contributions to implementation science, many questions remain, particularly regarding implementation strategies that increase the reach of evidence-based practices through inter-organizational linkages. First, there is a need for understanding whether and how implementation strategies may be applied to criminal justice systems, which are major stakeholders and purchasers of substance abuse treatment in the US (Kubiak et al. 2009; McCarty and Chandler 2009; Taxman et al. 2009). Criminal justice agencies operate under a fundamentally different mission and mandate (i.e., public safety) than their public health-oriented counterparts in the treatment system. Thus, testing an implementation strategy aimed at promoting inter-organizational linkages allows for a novel examination of the extent to which such disparate systems are amenable to coordination and cooperation. Second, it is unclear whether implementation strategies can achieve sustainable changes in inter-organizational processes, such as changes in patterns of referrals between organizations. The MATICCE study sought to address these gaps by testing an organizational linkage intervention (OLI) to improve collaboration and coordination between correctional and treatment systems and thus promote offender access to evidence-based treatment.

Methods

Sites and Participants

The MATICCE study involved nine research centers (for further details, see Friedmann et al. 2013). Each Research Center collaborated with two or more probation/parole agencies located in divergent geographic catchment areas. A ‘study site’ consisted of the probation/parole agency and at least one local treatment agency currently providing MAT or willing to consider doing so. There were 20 study sites in total across 11 states and territories including Arizona, Connecticut, Delaware, Illinois, Kentucky, Maryland, Missouri, New Mexico, Pennsylvania, Rhode Island, and Puerto Rico. Sites were block randomized within each jurisdiction, with each research center contributing one or more pairs of experimental and control sites. Study participants included probation/parole personnel at various organizational levels (e.g., officers, unit supervisors, and directors), and both medical and non-medical treatment personnel (e.g., substance abuse counselors, nurses, and medical directors).

Description of the Organizational Linkage Intervention

Each of the research centers collaborated to design the OLI activities and produce a well-structured manual intended to help guide and standardize the activities and benchmarks across different sites. The intervention focused on structured communication between the probation/parole and treatment agencies through a local change team. Implementation strategies involving the use of local change teams have demonstrated effectiveness in increasing the use of evidence-based practices in service delivery (Capoccia et al. 2007; Edmonson 2003; Fixsen et al. 2005; Lehman et al. 2009; McCarty et al. 2007). The local change team included staff from the treatment and probation/parole agencies as well as other key agencies linked to MAT treatment.Footnote 1

Across all sites, each change team had 8–10 participating members. The local change teams included executive staff from probation/parole (e.g., Chief and/or Deputy Chief of Probation) and treatment agencies (e.g., Executive Director and/or Assistant Director); middle management from probation/parole (e.g., Drug/Mental Health Court Supervisor, Intermediate Punishment Program Director, DUI Program Supervisor) and treatment agencies (e.g., Clinical Director, Clinical Supervisor, Head Nurse), and at least 2–3 line staff from probation/parole (e.g., probation officers) and treatment agencies (e.g., nurses and counselors). Each change team selected two co-chairpersons with decision-making authority, one from the probation/parole agency and one from the community treatment agency. Co-chairs were charged with setting agendas for change team meetings and facilitating discussion, decision-making, and planning among change team members. Each change team was also supported by a Connections Coordinator (i.e., boundary spanner) whose role was to facilitate the group’s activities and problem-solve in the area of inter-organizational coordination.

Each change team participated in a 4-phase process. The purpose of the Assessment phase was to inventory the existing policies and procedures at both the probation/parole and community-based treatment agencies surrounding assessment, referral to treatment, and MAT for adult offenders. The change team identified policies and procedures that might constrain or facilitate the referral and assessment of clients who are eligible for MAT. During the Strategic Planning phase, the change team developed a detailed strategic plan using the findings from the Assessment phase. Each change team was directed to identify and operationally define 3–5 objectives they wished to target from three process improvement domains: (1) building relationships between probation/parole and community-based treatment agencies that provide MAT, (2) getting buy-in from line staff for MAT, and/or (3) financing MAT. Objectives could include tasks such as training and education, reassigning staff, or developing new procedures for cross-agency collaboration and information sharing. The major task during the Implementation phase was to carry out the tasks and actions outlined in the strategic plan. During this phase, change team members were permitted to revise existing goals based on barriers they encountered while implementing the plan. This 6-month phase was considered complete when all objectives specified in the plan had been achieved, or when there was mutual agreement by members of the change team that the maximum progress toward their goals had been attained. The purpose of the Follow-Up phase was to bring closure to the initiative, institutionalizing those actions needed to assure the sustainability of any changes implemented during the preceding phase. Development of sustainability plans focused on plans for continuing the change team and/or further enhancing IORs.

Fidelity (completion of major activities within each phase and the duration of each phase) was monitored closely through a monthly checklist completed by researchers at each site and submitted to the MATICCE Executive Committee, which included the NIDA Program Official and four Principal Investigators. Any deviation from prescribed procedures, activities, or phase durations required an explanation and a plan of correction. At the conclusion of the study, there was no variation in milestones completed and little variation in duration of phases. On-site researchers also participated in regular conference calls (weekly during the first 6 months, then bi-weekly thereafter) to discuss implementation and related issues. Examination of the strategic plans for each site indicated a good deal of similarity in strategic goals across experimental sites. Nine of the ten sites attempted to strengthen interagency communication and information sharing protocols; seven sites conducted some kind of cross-trainings involving both probation and treatment personnel; and four sites focused on expanding sources of MAT funding (e.g., local, county and state government; private and non-profit). All sites targeted at least two of these three primary goals.

Data Collection Procedures

This study drew upon both quantitative data from surveys and qualitative data from semi-structured interviews. All staff and administrative personnel participating in the study provided written informed consent. Prior to randomization (Time 1), study participants completed (in person or by mail) a Baseline Survey of Organizational Characteristics (BSOC) and an IOR survey. IOR surveys were repeated at the end of the intervention 12 months after startup (Time 2). In both experimental and control sites, confidential recorded interviews were conducted with four probation/parole staff at baseline. These respondents were selected randomly from those who participated in both surveys and the MAT training at each site, although researchers attempted to sample from a range of roles (i.e., staff, supervisors) within the organization. In experimental sites, additional interviews were completed with four members of the change team at both baseline and follow-up. The change team interviews included two members each from probation/parole and the treatment agency.

Measures

Dependent Measures

A 20-item IOR survey was based on a reliable, well-validated instrument used to assess dyadic relations between human service organizations (Van de Ven and Ferry 1980). Each participating agency was asked to rate the other, i.e., probation/parole staff rated the treatment agency, and treatment staff rated probation/parole. If a site had more than one treatment agency participating in the change team, probation/parole was asked to rate each treatment agency, and each treatment agency was asked to rate probation/parole. Wording of questions varied slightly in each version (e.g., the referent agency). Five dimensions were assessed; most items were worded as five-point Likert scales (e.g., 1 = Not at all; 5 = Very much). Resource Dependence (5 items, α = .83) assesses client referrals, information exchanges, and funding flows between agencies. For example, one item asks: “To what extent does probation/parole send clients with alcohol or opioid problems to the local treatment provider?” Perceived Effectiveness of Relationship is a 4-item scale (α = .94) that assesses how productive and worthwhile the respondent feels the relationship with the other agency is. One sample item asks: “To what extent do you believe the relationship between probation/parole and this treatment agency is productive?” Agency and Personal Awareness is a 3-item scale (α = .87) that asks the respondent how familiar they are with the other agency and its personnel. For example, one item asks: “How well informed are you about the specific goals and services that are provided by this treatment agency?” Quality of Communications (3 items, α = .67) asks the respondent to rate how easy it is to reach staff in the other agency and how useful such interactions are. One item asks, for example, “When you have wanted to communicate with persons in this treatment agency, how much difficulty have you had in getting in touch with them?” Frequency of Communications (5 items, α = .84) asks the respondent how often they have had different types of communication (e.g., phone, e-mail, face-to-face) with personnel in the other agency. All items in this subscale employed nine-point Likert scales (0 = Zero times during the past 6 months; 1 = One time during the past 6 months; 2 = Two times, or about every 3 months; 3 = Three times, or about every 2 months; 4 = About every month, or six times; 5 = About every 2 weeks, or twelve times; 6 = About every week, or 24 times; 7 = About every 2–3 days; 8 = About every day).

Independent Measure

The independent variable was Study Condition (experimental vs. control). Although randomization in theory equalizes groups on diverse characteristics that might influence the dependent variables, the total number of sites (n = 20) in this study was insufficient to assume such equivalence. Relevant site-level covariates were thus identified to equalize the experimental and control groups on organizational characteristics known to influence the results of structured change efforts (e.g., Aarons et al. 2011; Greenhalgh et al. 2004; Lehman et al. 2002; Proctor et al. 2009).

Covariates

The BSOC provided a descriptive context for the organizational structure and climate of different agencies participating in the study. BSOC scales were derived from the TCU organizational readiness for change (ORC) and survey of organizational functioning (SOF) instruments (Broome et al. 2009; Lehman et al. 2002). The BSOC included twenty-nine scales organized into five sections: (a) Needs/Pressures for Change; (b) Resources; (c) Staff Attributes; (d) Organizational Climate; and (e) Other (e.g., Support for Evidence-Based Practices). Demographic information (e.g., age, race, ethnicity, gender, work experience, job characteristics) was also gathered through the BSOC survey. In addition, attendance at the three-hour MAT training session (1 = Yes, 0 = No) was controlled for in analyses, as trainings at each site prior to randomization may have provided a potential platform for initiating or enhancing interagency relationships.

Interviews

Qualitative data were collected via semi-structured interviews designed to probe further the contours of stakeholders’ (i.e., probation, treatment providers, funding agencies) IORs, communication patterns, and perceptions of barriers and facilitators to MAT. Time 2 interviews were designed to capture potential change over time in respect to the nature and quality of IORs. The semi-structured interviews allowed some flexibility for interviewees to raise issues unanticipated by interviewers or to elaborate on themes important to the respondent.

Analyses

Quantitative Analyses

It was hypothesized that probation/parole agencies and community treatment providers that engaged in a structured, 12-month OLI would show a greater increase in inter-organizational service coordination (Resource Dependence, Effectiveness of Relationship, Agency and Personal Awareness, Frequency of Communications, and Quality of Communications) than units receiving a three-hour MAT training session alone. Hypotheses required examination of changes over time (baseline and follow-up) in the experimental versus control sites. Mixed effects models, also known as Hierarchical Linear Models (Hedeker et al. 1994; Raudenbush and Bryk 2002) using repeated measures were examined using SPSS version 21 generalized linear mixed models (GLMM). Mixed effects models account for the covariance structures for between-clusters as well as within-clusters, since perceptions of IORs were measured at two time points, and staff were nested within clusters (sites and research centers).

To the extent possible, the same subjects were sampled at both time periods, and the brevity of the intervention (12 mos.) facilitated our ability to do so. Having the same measures on all respondents for both time periods could reduce the within-subjects error term (rho) and increase the statistical power of analyses. However, mixed effects models do not require a balanced design (i.e., mixed effects models do not assume an equal number of observed occasions for all participants, and allow the use of all available cases when estimating the effects), whereas traditional multivariate analyses of variance will only analyze cases that have complete follow-ups without missing observations, which is not realistic in longitudinal studies. For both cohorts, 3-level HLM models were examined with random effects estimated for research center (Level 3) and site (Level 2).Footnote 2 Minimum detectable effect size (MDES) was calculated for a 3-level MSCRT (Spybrook and Raudenbush 2009); there was at least 80 % power to detect a true population effect as small as 0.37, corresponding to a medium-small effect size (see Friedmann et al. 2013).

Sequential Bonferroni planned contrasts were conducted to examine mean differences for group × time interactions (e.g., did the experimental group improve more on Quality of Communication than the control group)? As opposed to overall (omnibus) F-tests for interaction terms, planned contrasts isolate the appropriate group x time means and provide more statistically powerful tests (Maxwell and Delaney 2004). To control for possible site-level differences that may have compromised randomization procedures due to the small number of sites (j = 20), site-level differences between the experimental and control sites on demographic and organizational variables were examined, and only measures that significantly distinguished experimental and control sites were entered as site-level covariates in HLM analyses. Model fit was examined by inspecting observed versus predicted residuals and examining goodness of fit indices (e.g., -2LL, Akaike, Bayesian).

Qualitative Analyses

The purpose of the qualitative analyses was to help explain the quantitative findings and to illuminate subtle dynamics that may have been latent in, but not adequately captured, through survey measures. Overall, the quantitative and qualitative data played primary and secondary roles respectively, captured simply as “QUANT-qual” (see Palinkas et al. 2011). Following this approach, the function of qualitative analyses was “expansion”—to provide context and identify possible interpretations for the quantitative findings. Audio file transcripts were transcribed into text documents. To protect confidentiality, the names of respondents as well as agencies were redacted. Redacted transcripts were then uploaded into Atlas.ti (v. 6.2) for qualitative coding. Transcripts were coded using a universal scheme that was created through an iterative consensus process among a cross-center workgroup. This process resulted in a standardized codebook that contained a list of codes that was agreed upon by qualitative analysts from each center. Using this codebook, at least one researcher from each center coded the Time 1 and Time 2 transcripts using Atlas.ti. Centers with multiple coders worked internally to perform coding reliability checks.

Analyses consisted of a careful reading of all text segments coded under primary code categories deemed relevant to the project hypotheses. The analysts wrote memos concerning any thematic issues and conceptual relationships emerging from the data. At specific intervals, the analysts merged their memos and compared their insights. Following this initial exploration, a team of three analysts examined specific sets of data (identified below) from a large cross-site dataset (hermeneutic unit) which contained coded data from both T1 and T2 intervals (n = 207 interview transcripts). Each of the three analysts examined data from two of the following six categories of interviews (known as ‘families’ within Atlas): (a) all Connections Coordinators; (b) all treatment providers; (c) change team members at baseline; (d) change team members at follow-up; (e) baseline survey respondents; and (f) follow-up survey respondents. Within each family, the main codes that were the focus of analysis were “current IOR,” “IOR enhancement,” and “personal views of other agency.”

Results

Quantitative Results

The survey response rate for the probation/parole cohort was 80.1 % (793 forms returned/990 forms distributed). After exclusion of records from respondents who were recorded as being in “neither” or “both” study conditions (7 records), duplicate cases (1 record), and those missing critical identifiers such as Interval or Treatment Agency Rated (4 records), a total of 781 IOR1 surveys from 439 probation/parole respondents were available.Footnote 3 The mean age of the probation/parole sample was 44.0 years; 39.4 % were male; 25.8 % were African American; and 19.8 % were Latino/a. The response rate for the Treatment Provider cohort was 81.2 % (411 forms returned/506 forms distributed). After exclusion of surveys from respondents who were listed as being in “neither” or “both” study conditions (24 records), duplicate cases (2 records), and those missing critical identifiers such as Interval (1 record), a total of 384 IOR2 surveys were obtained from 270 Treatment Provider respondents.Footnote 4 The mean age of the Treatment Provider cohort was 49.0 years; 30.9 % were male; 15.4 % were African American; and 17.6 % were Latino/a.

None of the potential covariates distinguished experimental and control sites for the probation/parole sample (Table 1). For the treatment provider sample, experimental and control sites differed significantly only on Attendance at Training (p < .032) and one organizational characteristic assessed by the BSOC survey, Influence (p < .036). Influence (6 items, α = .82) refers to the perceived ability of the respondent to influence their coworkers. The procedures for randomly assigning sites to experimental and control conditions were thus highly successful with these two exceptions; therefore, both variables were entered as site-level covariates in HLM analyses. While the research design neither predicted nor required equality between the probation/parole and treatment cohorts, several differences between the two cohorts are worth noting. For example, a slightly greater proportion of probation/parole respondents were African American and male, while a greater proportion of treatment providers had a post graduate degree. Probation/parole respondents tended to be slightly younger, had been at the same employer for a longer period of time, and had a higher active caseload than treatment respondents. Differences in organizational cultures are addressed in the qualitative results.

Table 1 Site-level demographics for probation/parole and treatment agencies
Full size table

HLM Results for Probation/Parole Cohort (IOR1)

A 3-level HLM model was used, where L3 = research center (j = 9), L2 = site (k = 20), and L1 = individual (n = 781). Fixed and random coefficients for HLM analyses are presented in Table 2. Estimated means and significance tests for the planned contrasts, used to test the main study hypotheses, are presented in Table 3. Contrary to hypotheses, the experimental group showed no significant differences from the control group on any of the five outcomes examined for the probation/parole cohort.

Table 2 HLM results with planned contrasts
Full size table
Table 3 Planned contrasts: estimated means and significance tests
Full size table

HLM Results for Treatment Provider Cohort (IOR2)

A 3-level HLM model was again used, where L3 = research center (j = 9), L2 = site (k = 20), and L1 = individual (n = 384). Compared to the control group, the experimental group showed significant improvement on two outcomes: Agency and Personal Awareness, and Frequency of Communication (Table 2). No other contrasts were statistically significant. At level 2 (site), neither Influence nor Attendance at Training significantly predicted any of the five dependent variables.

Qualitative Results

Given that the qualitative data played a secondary role in the design of the study, the findings below are suggestive, rather than definitive. Qualitative results are organized around three core themes: (1) the need for greater involvement of line staff in the change process; (2) a slower, but more realistic pace of change; and (3) differing expectations about intervention outcomes.

Involvement of Line Staff

Although many respondents noted the benefit of cross trainings that involved line staff (i.e., those with client caseloads), intervention effects might have been stronger if line staff had greater involvement and ownership throughout the intervention. One treatment agency administrator makes this point:

… the key is always getting things down to front line staff. You know, making changes at an administrative level, us being on board. That’s all great but a lot of times it doesn’t trickle down to, you know, where the real resistance and communication is, and that’s why cross-training I think was good because that did involve front line staff. I think that’s what was great about that, and yes, we had an opportunity to really improve our relationships with administrative people throughout the system. And that was great, but many of our staff didn’t have those same opportunities, other than cross-training.

Pace of Change

Other subtle changes may have occurred in experimental sites, although stronger interagency communication may take additional time to develop. As one change team member (with a county agency other than probation/parole) expressed in a follow-up interview:

Our organization’s current direct referrals are probably pretty limited because of the limited number of providers, as well the limited education. I think we’re at the ground floor right now, of establishing relationships and really looking at bringing our staff up to par in their knowledge of how to share this information with their clients, what an ideal client would look like, or even making the referral or the recommendation and just starting to build the relationships now with treatment providers, the few treatment providers in our area that do provide MAT.

At the baseline period, one treatment provider expressed an equally cautious view about the pace of change:

Well… I wouldn’t even say now that there’s an established relationship. I mean we’ve been working on it for years. Just recently over the last month, I’ve put somebody out on, you know on-site at [community corrections] and I’ve started working closely with [community corrections officer] and some other workers, so I think it took years to get there and I think this new, what do you call it, the [change team], this new thing that we’re on, I think is kind of sealing the deal, but I still would not say that we have a wonderful relationship.

Differing Expectations

Qualitative data provided several possible explanations for why treatment personnel perceived greater improvement in IOR than probation/parole personnel. First, treatment and probation personnel may have had different expectations, from the outset, of what could or should be achieved from the intervention. For instance, for reasons of privacy (Health Information Portability and Accountability Act—HIPAA) and ethics, treatment providers often had tighter boundaries regarding what information could be shared. Probation/parole personnel who were not on the change team may not have been fully aware of these limitations, so that they blamed failure to share information on the treatment agency. For example, one probation officer believed that a treatment agency should have given her urinalysis results in order to prevent the client from having to pay for a separate test:

They won’t release drug screen information to me even though we’ve had an understanding. I’m not trying to jam this person up. I just don’t want to send them. I don’t want them to spend another $16.00 to do drug screens for me when you’re already sending them to do drug screens for you. If we could just share that information and the cost is covered by them. So, if I can save these folks $16.00, you know, quite realistically that’s a choice between do I buy my kid formula this week or not.

A probation officer in a control site commented on how treatment staff members were “afraid” of HIPAA. When asked about the strength of their agency’s relationship with treatment agencies, he/she indicated:

I would still say it’s not really that strong. There are multiple providers that do Methadone and Suboxone. A lot of them are pretty reluctant to share information because they feel like HIPAA is a huge, huge deal, and they’re just so afraid of HIPAA. It could be—it could definitely be better.

Members of the change team from probation/parole believed that there was more treatment agencies could do to “loosen” restraints on sharing information between agencies. At one site, the change team created a single release of information form that could be exchanged readily between treatment and criminal justice agencies. The form was approved by one of the treatment agencies, but not by the other, because of remaining concerns about client confidentiality. As the Connections Coordinator of this change team mentioned:

I wished the form would have been approved by everybody. I was still pleasantly surprised I guess is the right word with [Treatment Agency] at least giving– saying that they were going to kind of play ball in that same park in terms of getting– we’re not going to use one form. We’re going to use the five forms. We’ve always used the five forms. We’re going to use the five forms. It’s kind of holding, I don’t want to say holding their fingers to the fire, but kind of, are you really going to do that? And, there’s no way of really monitoring that or forcing the issue ….

Conversely, some change team members from the treatment sector felt there was little they could do beyond current practices. One team member from a treatment agency responded to an inquiry about current agreements surrounding the sharing of client information between agencies:

I think that most of the things that I saw, like even with the few conflicts that happened within the meetings… that has to do with stuff that can’t be touched. Like, things that we can’t change and things… that they didn’t understand about the way things have to be done. You know not - within our organization, but because of the whole nature of drug and alcohol treatment, method of treatment that… you know, we can’t even change. Like it seemed like those were the things that really, and even when we were doing the training there, you know, how they were like - you can’t even blah blah blah…. And we’re like, no! You can’t! It seemed like they- some of the wedges… between the whole thing … like with the confidentiality in general. You know, that’s not necessarily [MAT Treatment Provider], that’s the nature of the whole thing. So… yea I think that it’s not necessarily something we can do anything about. I mean maybe them understanding a little bit more of why, but I don’t even know if you can do that.

Differences in organizational culture and professional role perceptions were evident in some of the responses. Treatment providers appeared to place a greater value on enhanced education and awareness of roles across agencies than did probation/parole staff. To many treatment providers, it seemed that clarifying the nature of structural barriers to cooperation was, in and of itself, an important accomplishment of the change team. The change team process helped address numerous misconceptions, as one respondent explained:

I think that in the past, you know at least personally and from what I hear from my colleagues here, is just the feeling of being bullied in a way by probation or other law enforcement, you know? Like trying to be pushed when we really, we can’t be doing anything. So it’s like… and just them not understanding why, you know, we couldn’t do what they wanted us to do basically. So, you know, kind of explaining – letting them know why we can’t do those things. What was holding our hands behind our back about certain things, you know? That… is positive.

It seemed important for treatment providers to be better understood by their corrections counterparts in terms of their roles and restrictions. At baseline, when asked about the issues they hoped to see addressed through the change team, a treatment provider said simply, “I think…the best hope for me would be for people to understand what we do”. When asked if anything had changed in terms of relationships with a particular community corrections agency following the intervention, a provider responded, “[p]robably the number one helpful thing for me from the whole project was our relationship with [EXPERI COMMUNITY CORRECTIONS AGENCY] is much better, within MAT and outside MATs.” Treatment providers expressed a better understanding of mutual roles with probation/parole agencies through cross-training. One treatment respondent stated: “[i]n communication and understanding I think the cross-training was phenomenal.” Treatment personnel learned about the challenges of working in a criminal justice environment and saw this mutual learning about different organizational cultures as valuable:

Criminal justice is very different, and I’ve learned that in working with them is that you know we come with very different goals in mind a lot of times and different understanding of how to get to those goals. And, so you know but I think with the change team it’s kind of like bridging some of that and in that group. I mean, we’ve kind of begun to really be able to understand each other and stop criticizing each other…. I think we’ve kind of learned that, you know, we’re all trying to do similar things with the population, and all of us have different challenges that impact our ability to do things, whether that’s funding or whether that’s at the administration or whatever you know it’s just different things and so I think that has been good about the change team.

A treatment respondent in another site stated that the relationship with probation had become “richer on both ends” after the project started. They later explained, “I would say that I’ve certainly brought from the committee the information that’s been shared, the clarification of probation’s goals for our patients and their patients. And I’ve been able to clarify with probation what our treatment is about.”

Discussion

Conceptual models of implementation (e.g., Aarons et al. 2011; Proctor et al. 2009; Damschroder et al. 2009) argue that successful implementation of evidence based practices requires a coordinated, interagency effort to address service gaps experienced by shared clients. Few studies to date, however, have explicitly examined IOR between probation/parole agencies and community-based treatment providers as implementation outcomes. The MATICCE study hypotheses were partially supported, as treatment personnel perceived greater improvement on two key interagency dimensions: (a) Agency and Personal Awareness, and (b) Frequency of Communications. Contrary to study hypotheses, however, probation/parole personnel in the experimental sites perceived no significant improvement in interagency relationships.

In part, the different perceptions of probation/parole personnel and treatment providers reflected differences in organizational structure and culture. For example, treatment agencies tended to be smaller than the correctional agencies (mean number of full-time employees employed by facility’s parent organization = 218.9 and 1498.7, respectively), which may have facilitated intra-agency diffusion of information. The two types of agencies evidenced greatly differing norms about privacy and confidentiality. On one hand, probation/parole personnel, who were not constrained by the HIPAA Privacy Rule or Chapter 42 of the Code of Federal Regulations (42 CFR), were often frustrated by the privacy concerns of treatment providers. On the other hand, overzealous privacy concerns among some treatment providers may have inhibited legitimate information sharing with probation/parole agencies (see Petrila 2007). Sharing of information is central to effective coordination, continuity and integration of services for individuals who receive services from both criminal justice and health sectors (Etten and Petrone 1994).

The organizational cultures of correctional agencies tend to be associated with their use of evidence based treatment practices. Several studies reported that utilization of evidence based practices by adult offender treatment programs was related to more extensive networking relationships with various corrections and community agencies; performance-oriented cultures; non-punitive cultures; climates conducive to learning; resources devoted to training; and directors who view rehabilitation as a central goal of the criminal justice system (Friedmann et al. 2007; Henderson et al. 2008, 2009). Directors in agencies that utilized a wider array of evidence based practices also tended to have a human services background, a high regard for the value of substance abuse treatment, and a wider knowledge of evidence based practices. Utilization was also more likely in states that had more integrated criminal justice-health services agencies and more stable and adequately staffed executive agencies with executives who placed a high importance on corrections-based substance abuse treatment (Henderson et al. 2009). Organizational culture, therefore, may be important at the state level as well as the agency level.

The differing organizational cultures of treatment and probation/parole agencies are reinforced through structures of education, training and socialization. Probation and parole officer “professional orientation,” for example, refers to attitudes toward offenders and interactions with offenders (Whitehead and Lindquist, 1992). Role conflict in probation/parole agencies is often attributed to inconsistencies in three main functions of offender supervision: (a) to enforce the legal requirements of supervision (the “law enforcement” role), (b) to assist the offender in successful community adjustment (the “social worker” role), and (c) to carry out the policies of the supervising agency (the “bureaucrat” role) (Clear and Latessa 1993; Hepburn and Albonetti 1980). While some believe that role integration is possible and desirable, others believe that role conflict is inevitable (Sigler 1988) and role conflict increases the likelihood of burnout, absenteeism, and turnover (Whitehead 1984). Agencies vary in their support for different role orientations, and organizational culture is often a key influence on professional role orientations (Clear and Latessa 1993).

Stronger mechanisms to build and sustain ongoing relationships between change team and non-team members may be useful in future implementation interventions. In MATICCE, a slight majority of agency representatives on the local change teams were in executive and supervisory positions, relative to line staff such as probation/parole officers and treatment counselors. Part of the strategy in this intervention was to have key agency decision makers directly involved in the change teams. In surveys and interviews, however, where a greater number of line personnel in each agency were sampled, results suggested a need for line staff to be more directly involved in planning and implementing changes. More balanced and meaningful involvement by line staff will likely require careful attention to latent power dynamics among agency personnel, regardless of organization type (Aime et al. 2014). For example, including line staff along with supervisors on change teams or in cross trainings does not necessarily guarantee that line staff will openly voice their opinions or share their experiences. The concept of a power “heterarchy” is a conceptualization of power structures in groups that is more dynamic and fluid than traditional hierarchical structures. Heterarchical structures in which the expression of power actively shifts among team members to align team member capabilities with dynamic situational demands enhances team member engagement and creativity (Aime et al. 2014). However, this positive effect of power heterarchies is contingent on the team perceiving the shifts in interpersonal power expressions as legitimate. Three main implications for change teams follow.

First, managers need to adapt their understanding of which resources (e.g., expertise, information, etc.) are needed for a given situation. The focus should be on ensuring that change team members provide the most relevant resources needed for the change attempt, and are not appointed simply by virtue of job title or status. Second, as situational demands change, teams need access to a wide range of resources, such that the team’s resources can be aligned with changing situational demands through dynamic shifts in power expression among members. Finally, managers should nurture a team culture in which shifts in power expression are not only free to occur as situational demands change, but also will be seen by the team as legitimate. This recommendation implies that the reward structures should be designed and monitored so that team members do not simply vie for resources (e.g., trying to be seen as the expert), but will be more likely to see shifts in power expression as legitimate ways to improve team performance (Aime et al. 2010; Beersma et al. 2003; Johnson et al. 2006).

Other useful strategies might focus on exploring and setting mutual goals between probation/parole and treatment providers. A recovery-oriented system of care (ROSC) is defined as a coordinated network of community-based services and supports that is person-centered and builds on the strengths and resilience of individuals, families, and communities to achieve abstinence and improved health, wellness, and quality of life for those with or at risk of alcohol and drug problems (SAMHSA 2014; Sheedy and Whitter 2009). Principles include systems of care anchored in the community, integrated services, continuity of care, and partnership-consultant relationships (i.e., a model that focuses on collaboration rather than hierarchy) (Center for Substance Abuse Treatment 2007). The National Reentry Resource Center (NRRC) promotes ROSCs as a means of improving service coordination for justice-involved clients with substance use disorders and offers a number of recommendations to enhance communication between health and justice agencies (NRRC 2011). Partners for recovery (PFR), a SAMHSA initiative, has developed several resource guides to disseminate information about ROSC to broad audiences (SAMHSA 2014).

Professional associations such as the American Probation and Parole Association (APPA 2014) and the American Association for the Treatment of Opioid Dependence (AATOD 2014) also play an important role by disseminating policy statements that seek to reduce barriers to MAT and improve systems coordination across health and criminal justice sectors. Trainings on MAT offered by the Substance Abuse and Mental Health Services Administration (SAMHSA 2013) and the Bureau of Justice Assistance (Miller 2013) also help address constraints to systems coordination across health and justice sectors.

The differences in perceptions of IOR across probation/parole and treatment agencies imply that a strong action research component—where external researchers and practitioners develop a shared understanding of desired outcomes and change mechanisms—merits consideration as a critical precursor to the implementation of a randomized experimental design (Welsh 2006; Welsh and Harris 2012). Such formative work could also help alert outside researchers to potential intra- and inter-agency factors that may influence the uptake, utilization, and sustainability of specific evidence-based practices (Aarons et al. 2011; Proctor et al. 2009). Future efforts to build and sustain effective IORs should not only bring diverse stakeholders together, but also develop explicit mechanisms for stakeholders to inform and engage each other in ongoing dialogues (see for example Salerno et al. 2011).

Limitations

Several limitations in the study design should be noted. First, the relatively small number of sites (j = 20), which was a function of the size of the national cooperative, limited somewhat the power of statistical tests. For example, we could not enter many level-2 (site) covariates in HLM analyses (Raudenbush 1997; Spybrook and Raudenbush 2009). While randomization of sites to experimental and control conditions was highly successful (as demonstrated by the reported finding that only one organizational subscale, Influence, distinguished experimental from control sites), the possibility that other, unmeasured site-level variables may have influenced the results cannot be ruled out.

The types of measures used to assess IOR carry specific tradeoffs and need to be carefully tailored to the purposes of the particular study. For example, the well-validated, dyadic measures used in this study allowed us to examine relationships between community correctional agencies and local treatment providers over time, but dyadic measures can be somewhat burdensome. Any one respondent, for example, may be asked to complete ratings of multiple agencies (e.g., more than one local treatment agency at most sites). Dyadic measures, however, allow us to control for potentially important individual-level variation in ratings within a given organization (Van de Ven and Ferry 1980). Other types of measures, including social network analyses (SNA), are also possible (e.g., Varda et al. 2008). SNA measures typically sample only one or two key respondents who are assumed to be knowledgeable about an agency’s relationships with other organizations in the environment (e.g., trust, reciprocity, and power). SNA measures, however, typically ignore line staff perceptions in favor of executive or supervisory personnel. In future implementation studies, researchers should carefully weigh the pros and cons of different types of measures to assess interagency relationships.

Further attention to fidelity may also be important. Implementation activities were all manualized, the progress of each site was monitored through monthly and phase reports, and there was a good deal of similarity in the strategic goals implemented across sites (Friedmann et al. 2013). It is still possible, however, that some variation in implementation occurred across sites and across phases of the intervention. Additional fidelity data (e.g., narrative reports submitted by each site at the end of each phase; qualitative analyses of working relationships between change team members) are currently being coded and analyzed as part of the larger CJDATS study.

A final limitation relates to the length of the intervention. Given the weak ties observed at baseline between many probation/parole and local treatment agencies, it may take longer than 12 months for strategic improvements in IORs to occur. The requisite trust and reciprocity for productive relationships may have been enhanced by the intervention, but increased face-to-face interactions between line staff, as well as a clearly perceived need for interagency communication may be needed to further enhance interagency relationships. Findings from the National Implementing Evidence-Based Practices Project suggest that successful implementation of evidence based practices is also related to strong agency leaders who support implementation efforts through workflow design as well as ongoing monitoring, feedback, and reinforcement (Torrey et al. 2012).

Conclusions

Future attempts to encourage uptake and penetration of evidence based practices should formulate implementation strategies at multiple levels of each agency and encourage involvement and engagement of diverse personnel with resources relevant to the change effort. Such an approach is consistent with management theory (Hitt et al. 2007; Ferlie and Shortell 2001) and emerging models of implementation science (Aarons et al. 2011; Chamberlain et al. 2008; Proctor et al. 2009; Tabak et al. 2012). The MATICCE study demonstrates the complexities of conducting real-world implementation research, particularly when the goal of that research is to improve the IORs between systems with deeply ingrained and divergent missions. By bringing both community correctional agencies and treatment providers to the table to engage in strategic planning activities, the intervention helped situate evidence-based treatment as a matter of both public health and public safety. Like the ACCESS project that inspired its design, the MATICCE study yielded limited support for hypotheses regarding improvements in IOR. Qualitative data, however, suggested that the OLI may have initiated a process of slow but steady change by initiating or increasing interagency dialogue. Further research is needed to identify necessary intervention components and participants that are best able to promote systems change while ensuring effective rehabilitation of offender populations.