Introduction

Dual Diagnosis

Over the past 15 years the prevalence and consequences of serious co-occurring mental health and substance abuse problems have drawn increasing attention from clinicians, researchers and policy-makers (Substance Abuse and Mental Health Services Administration, 2002). Epidemiological studies show lifetime prevalence of substance abuse or dependence among persons diagnosed with schizophrenia or bipolar disorder to be around 50% with current or recent rates in the range of 25–35% (Mueser, Noordsy, Drake, & Fox, 2003). Among those with serious mental illness, substance abuse is associated with relapse and hospitalization, disruptive behavior, familial problems, residential instability including homelessness, decreased functional status, HIV infection, medication non-compliance, higher suicide rates, hepatitis C, job interference, violence, abuse, and poverty (Drake & Brunette, 1998). Despite high prevalence and negative consequences, the 2002 National Survey on Drug Use and Health found only 12% of those having both types of disorder receive treatment (Epstein, Barker, Vorburger, & Murtha, 2004).

American reviews of the dual diagnosis literature have found evidence for integrated treatment rather than parallel or sequential treatment, (Drake, Mueser, Brunette, & McHugo, 2004; Greenberg, 2002; Sacks, 2000), but a Cochrane Review article (Ley, Jeffery, McLaren, & Siegfried, 2000) and a careful 2005 update (Donald, Dower, & Kavanagh, 2005), found insufficient evidence from randomized controlled trials to favor integrated treatment. All reviews have agreed that the methodological failings in extant studies are substantial. Despite these limitations, however, consensus panels in the past several years have endorsed integrated dual disorders treatment that includes assertive outreach, group and individual substance abuse interventions, motivational interventions, staged treatment, family interventions and comprehensive services—including housing and employment support (Health Canada, 2002; Drake et al., 2001). The particular version of dual diagnosis treatment developed by investigators at the New Hampshire Psychiatric Institute (Integrated Dual Disorders Treatment or IDDT) has been chosen by the Substance Abuse and Mental Health Administration to be one of six “evidence-based practices” it is promoting (Power & Demartino, 2004). An approach to implementing this treatment model is being tested in at least seven states, and a “fidelity scale” has been developed to measure achievement of the model’s principles (SAMHSA, 2005).

Co-occurring Disorders in Jail

Law enforcement agencies and advocates for the mentally ill have increasingly called attention to the fact that many jail inmates suffer from severe mental illness (Arboleda-Florez & Holley, 1988; Belcher, 1988; Lamb & Weinberger, 2001). Overall, persons with severe mental illness have been found to comprise between 6% (males) and 15% (females) of the jail population, and approximately 60% of the mentally ill (depending on the gender and mental health diagnosis) have a current alcohol use disorder while one third have a current drug use disorder (Abram & Teplin, 1991). Studies of persons with dual disorders in community programs have found high rates of law enforcement involvement (Mueser, Essock, Drake, Wolfe, & Frisman, 2001; Pandiani, Rosenheck, & Banks, 2003).

Studies conducted with participants selected for criminal justice diversion have lacked rigor, and findings have not been consistent. A recent federally funded program in multiple cities which compared over 600 diverted dual diagnosis clients with more than 600 not diverted (non-random assignment) found that after one year there were no differences in re-arrests (Broner, Lattimore, Cowell, & Schlenger, 2004) but that diverted clients spent less time institutionalized (Steadman & Naples, 2005). A rural program (without a control group) that provided both legal case management and dual diagnosis case management reported reductions in legal problems but had difficulty in accessing psychiatric services for a caseload made up primarily of persons with personality disorders (Godley et al., 2000). A small Chicago pilot using Assertive Community Treatment for 24 clients (with a high but unreported percentage of dual diagnosis) reports significant reductions in jail days (McCoy, Roberts, Hanrahan, Clay, & Luchins, 2004). In an uncontrolled study using the first 44 dually diagnosed clients served by Project LINK, a program combining ACT, a therapeutic community, and jail diversion, jail days were reduced by 50% after one year (Weisman, Lamberti, & Price, 2004). In a retrospective study, dually diagnosed clients who were diverted, when compared to non-diverted clients, had fewer days of incarceration in the subsequent year but only with reference to the more serious misdemeanors associated with longer jail stays (Hoff, Rosenheck, Baranosky, Buchanan, & Zonana, 1999).

Other studies have measured law enforcement outcomes in dual diagnosis programs having no formal diversion process. Clark and colleagues (Clark, Ricketts, & McHugo, 1999) presented three-year data for 203 clients in a program incorporating many of the IDDT principles discussed earlier. While, arrests decreased significantly overall, reductions were not found for persons with both bipolar disorder and substance use disorder (Drake, Xie, McHugo, & Shumway, 2004). A recent uncontrolled pilot program in San Diego based on IDDT principles reduced substantially the costs of arrests, convictions, incarcerations, and probation for 126 clients (Judd, Thomas, Schwartz, Outcalt, & Hough, 2003). In other studies, ACT was not effective in a randomized study of dually diagnosed prison parolees (Martin, Inciardi, Scarpitti, & Nielsen, 1997). And dually diagnosed clients in the uncontrolled ACCESS case management program for homeless mentally ill, while otherwise successful, increased jail utilization during the program (Gonzalez & Rosenheck, 2002).

In summary, the prevalence of dual disorders is high in treated, in untreated, and in jail populations. While a few diversion and some ACT or IDDT programs have documented the capacity to reduce arrests or jail days, other programs have not. The present study is the first randomized controlled trial of a high fidelity IDDT community-based after-care program.

Method

Program Description

California Senate Bill 1485 of 1998 (The Mentally Ill Offenders Crime Reduction Grant) established programs in 26 counties intended to reduce jail days and jail recidivism among persons with severe mental illness. (See Cosden, Ellens, Schnell and Yamini-Diouf, 2005, for findings from an ACT mental health intervention in one of these programs.) The experiment reported here focused on dual disorders and took place in Alameda County, a large urban county in the San Francisco Bay Area, between November 1, 2001 and June 30, 2004. Grant monies were used to fund an in-custody treatment unit and an intensive IDDT post-custody component. The in-custody component was provided to all study participants and included intensive assessment, medications, treatment planning in preparation for discharge, consultation to jail staff, one-on-one counseling, and crisis intervention. Post-custody care for control group members consisted in “usual services,” but also included the availability of up to 60 days of post-release grant-funded case management and housing assistance. Usual services included referral to one of the county-operated service teams for case management and medications. Although available, none of the control group members used employment or day treatment services.

Intensive IDDT post-custody services for the experimental group were provided for up to 2.5 years. The in-custody program, the brief aftercare, and the IDDT program were all run by the same agency in a coordinated way. The program had an advisory committee comprised of key county mental health and criminal justice administrators.

The program can be meaningfully described by showing the SAMHSA developed IDDT fidelity scale along with the rankings of the program on each item (Table 1). Items were rated two years into the program by the lead author, who was the project evaluator, and an independent psychiatrist well-versed in co-occurring disorders treatment. The maximum score is 5, the minimum 1.

TABLE 1 Integrated Dual Diagnosis Treatment Fidelity Scale Ratings of the Experimental Program: a Score of 1 is Low, 5 is High
Full size table

Mean program fidelity was 4.1 and 4.0 for the two raters. Inter-rater reliability, measured by kappa, was .82 (weighted so that ratings a point off are considered in 66% agreement; 2 points off are considered in 50% agreement). All team members had previously worked in substance abuse or dual diagnosis programs, and client to staff ratios were low: 35% of the clients were assigned to an ACT team (10:1 client to clinician ratio) while 65% received case management (20:1 ratio). In addition, a probation officer was assigned to the experimental clients and had an office at the site. She was the probation officer of record for about half of the clients, and she helped all case managers deal with the courts regarding their clients. Finally, the program had dedicated staff to assist in housing and to serve as substitute payees.

Selection and Randomization of Study Participants

Eligibility requirements for study participants included (a) both a current serious mental illness and a current substance use disorder, (b) not sentenced to prison, not on parole, and not a resident of another county, (c) not currently enrolled in another Alameda County treatment program, (d) Global Assessment of Functioning (GAF) score of 50 or less, (e) fluent in English or Spanish, and (f) having at least two jail episodes in the two years prior to the index admission (or having spent at least 90 days in jail in the prior two years). A Ph.D. level research associate interviewed potentially eligible inmates while incarcerated, determined the clients eligible to be randomized, and obtained a signed research informed consent.

The original design called for study groups of 100, assigned randomly. Accordingly, as potential participants were determined to be eligible over a 24 month period they were randomly assigned to the experimental or control group in blocks of two. After 13 months, in order to assure the fiscal viability of the IDDT program in the face of low enrollment (only 16 clients had been assigned to each study group), two experimental clients were assigned for every one control client (randomly in blocks of three). When 90 experimental participants were enrolled (and 48 controls), the assignment was reversed: two control for every experimental participant. This continued until 79 controls and 103 experimental subjects had been assigned. Due to this procedure, a greater number of the control group had study periods of a year or less after release from jail, but the total mean days at risk (days from initial release from jail until study termination date) were not significantly different: 572 for experimental and 556 for control participants (t-score = .571, p  <  .558; all tests are two-tailed).

Measurement and Analysis

At the initial research interview, the research associate administered two instruments: the PRISM is specially constructed to produce valid research DSM IV diagnoses in a dual diagnosis population (Hasin, Trautman, & Endicott, 1998); and the Circumstances, Motivation and Readiness Scale (CMRS) has demonstrated capacity to predict retention in treatment (De Leon, Melnick, Kressel, & Jainchill, 1994). In regression analyses, missing CMRS values for 17 participants were randomly imputed from among the non-missing values (Schonlau, 2003).

The research design utilizes administrative data, and no attempt was made to measure substance use or abstinence over time. Jail days, arrests, and convictions were obtained from the Alameda County Information Services Department. We have included only “original,” arrests, not subsequent arrests related to the original (due primarily to failure to appear in court). Mental health service units and costs were provided by Alameda County Behavioral Health Care Services. “Engagement” is measured by participation in outpatient services, particularly in the first 60 days after initial release from jail. “Engagement-related” services include outpatient, day treatment and vocational services. “High end” services are acute hospitalization days, psychiatric crisis visits, and long-term nursing home days. For experimental clients only, changes in functioning were measured during the study period with a 17 item case manager rating, the Multnomah Community Ability Scale (Barker, Barron, McFarland, & Bigelow, 1994).

Treatment costs are those used in the Alameda County mental health system and are based on California Department of Mental Health standards for cost accounting.

Bivariate and multivariate analyses are presented. To save space, only the comparisons of the experimental and control groups derived from multivariate models are presented in the text; the full models are available from the first author.

Results

Characteristics of the Study Participants

Consistent with randomization, experimental and control participants do not differ significantly on any of the demographic, diagnostic and clinical baseline variables in Table 2. Drug dependence was reported for 47% of experimental participants and 48% of control participants. For alcohol dependence the comparable figures are 31% and 37%. The overall mean on the CMRS treatment readiness scale for experimental clients was 72 and for control clients was 73.

TABLE 2 Demographic Characteristics and Initial Psychiatric Diagnosis of the Study Groups
Full size table

Criminal Justice Measures

Change from Baseline to Study Period

Because there was an intervention in the jail which both study groups received and one in the community which only experimental participants received, there are two research questions: do both groups show a reduction in arrest, convictions, and jail days (hypothesized to be at least in part a consequence of in-jail psychiatric services)? And, do the experimental participants have lower rates than do control participants during the study period? Table 3 presents the baseline and study period arrest, conviction and jail days data standardized to person years. The change in jail days from baseline to study period is significant for both groups. The change is not significant for felony convictions in either group. The change is significant for arrests and total convictions in the experimental group but not the control group.

TABLE 3 Baseline and Study Period Arrests, Convictions, Jail Days and Costs Per Person Per Year
Full size table

Experimental-Control Differences Regarding Time to First Re-Arrest and Total Number Of Re-arrests

Figure 1, following the display recommendations of Pocock, Clayton, and Altman (2002), shows the time from initial release from jail until the first re-arrest. The numbers along the bottom but inside the graph show persons still not arrested at each time marker. The difference between groups (relative risk .94) is not statistically significant (p  <  .75). In a Cox semi-parametric multivariate regression model, the study group difference remained non-significant (z  =  − .27, p  <  .79).

FIGURE 1
figure 1

Time between Initial Release from Jail and First Arrest Thereafter (Y-axis is Fraction of Group Experiencing a First Arrest; X-axis is Time in Days from Release from Jail until First Arrest)

Full size image

Figure 2 shows arrests for the study groups during the first 20 months after the initial release from jail (Ns were small after this point). We estimated a negative binomial regression model for the number of arrests during the study period (Long, 1997). The model incorporates days at risk (exposure), and we used the same covariates as in the survival analysis. Confirming Figure 2, experimental group members had a non-significant lower sum of arrests than did control participants (z  =  − 1.31, p  <  .189).

FIGURE 2
figure 2

Percentage Arrested in Each Month after Initial Release from Jail, by Study Group

Full size image

Convictions

Accounting for baseline convictions, time at risk and other covariates, the difference between the percentage of control and experimental participants having any convictions was not significant when estimated in a logistic regression model (mean of .6 per person year vs. .7 per person year, z  =  − .01, p  <  .989). However, when the convictions per person year among those who had any were estimated with truncated poisson regression, experimental participants were less likely to have multiple convictions (mean of 2.0 vs 2.6, z  =  − 1.93, p  <  .019). There was no difference between the study groups regarding felony convictions (see Table 3).

Jail Days and Incarcerations

The difference in jail days between the study groups was not significant (z = 1.16, p  <  .246). However, the experimental participants had significantly fewer incarcerations (mean of 2.2 vs. 2.8, z = 1.97, p  <  .049), which were balanced out by longer jail stays (mean of 59.4 vs. 43.3, t = 1.97, p  <  .051).

Mental Health Measures

Overall Service Costs

Overall mental health service costs are shown in Table 3. Service goals were to reduce inappropriate use of high-end services while increasing engagement and the consistent use of community-based services. Average mental health service costs per study participant for the high-end services stayed essentially the same for experimental group members (baseline = $2,903; study period = $2,978). For control group members average high-end costs increased from $647 to $8,216. Average engagement-related costs for experimental participants increased from $2,276 in the baseline to $11,787 in the study period; average engagement-related costs for control participants increased from $1,037 to $2,829. Figure 3 shows the percentage of participants with high end and engagement-related expenditures during the first 18 months after the initial release from jail.

FIGURE 3
figure 3

Percentage of Experimental and Control Participants having Engagement-related Services or High-end Service Costs in the 18 months after Each Person was Initially Released from Jail

Full size image

Psychiatric Hospitalization and Crisis Use

Psychiatric hospital days used by experimental participants decreased from a baseline mean of 1.54 (SD 4.59) to a study period 1.25 (SD 3.27), not a significant change (sign rank test, p  <  .667). Control participants days increased from a mean of .34 (SD 1.40) in baseline to 5.03 (SD 13.88) in the study period, a highly significant change (sign rank test p  <  .001). Modeled using logistic regression, there is a marginally significant difference in the study period favoring the experimental group regarding whether participants had any hospital days, controlling for baseline and days at risk (Likelihood ratio chi-square = 5.20, df = 2, p  <  .074). If hospitalized, however, the mean for the experimental group was 6.89 days (SD 7.01) vs. 12.52 (SD 16.88) for the control group (t = 2.46, p  <  .014 modeled with truncated negative binomial regression).

Crisis visits increased non-significantly for experimental participants (baseline mean 1.62, SD 3.56 vs. 2.10, SD 4.59; sign rank test p  <  .654). Control participants increased their crisis visits dramatically (baseline mean .58, SD 1.29; study period mean 3.32, SD 6.95; sign rank test p < .001). The study group difference between those having any crisis days (45% for experimental, 51% for control) and those having none during the study period is not statistically significant (modeled in multiple logistic regression z  =  − .64, p  <  .519) but the number of crisis visits among those with at least one was greater for control than experimental participants (incidence rate ratio of .301 modeled with zero truncated negative binomial regression z  =  − 2.12 p  <  .034).

Engagement and Retention in Outpatient Services

For experimental participants only, we have detailed information on initial engagement and on program attrition. Of 103 persons in the study group, 11 persons disappeared immediately upon release from jail and could not be found subsequently. Another 31 clients were lost to follow-up after receiving services for varying lengths of time, so total attrition from experimental treatment was 41%. Among those who were engaged and participated in the program, there was a 29% attrition rate if the period of time in the community after the initial release from jail was two years or under and 53% if over two years.

Both the in-custody and transitional services were designed to achieve engagement with an outpatient provider within the first 60 days, as was the experimental program, of course. A total of 77% of experimental and 18% of control participants did receive an engagement-related service within 60 days after leaving jail.

Medications Services

Receipt of psychiatric medications even while using alcohol or other substances is a central tenet of dual diagnosis treatment and may even help clients achieve abstinence (Mueser et al., 2003) or prevent recidivism (Farabee & Shen, 2004). Overall 83% of experimental and 62% of control participants registered receipt of some amount of outpatient medications services (chi-square = 10.76, df = 1, p  <  .001), including, respectively, 81% and 64% of those with a schizophrenia diagnosis and 79% and 33% of those with a diagnosis of major depression. The difference in mean hours of medication service received, if any, was also pronounced: experimental participants averaged 6.52 hours while control participants averaged 3.02 (t-score = 2.96, p  <  .004, OLS regression).

Institutional Days

We combined all institutional days—jail days, acute hospital days, and long-term nursing facility days. Overall, 73% of experimental and 73% of control participants had at least one institutional day (Chi-square = .008, df = 1, p  <  .928). Among those institutionalized during the study period, the mean days for experimental participants were 140.6 vs. 151.9 for control participants, not a significant difference (t  =  − .28, p  <  .776; OLS regression).

Changes in Functioning

As noted in the methodology section, no follow-up clinical interview was conducted with control group members. The Multnomah Community Ability Scale was administered (at least) twice to 47 experimental group clients well known to their case managers during periods when they lived in the community (another 24 clients who received only one rating did not differ significantly at first administration from those with multiple administrations (mean of 49 vs. 53, t test: t-score p  <   =  .15)). For the 47 persons with multiple scores at least a year apart, the first mean was 53.2 (SD 12.9) and the second 52.1 (SD 13.5)—not a statistically significant change in functioning (paired t test: t-score = .50, p  ≤  .621). The item which rated impairment due to substance abuse also did not change significantly (paired t test: t-score  =  − 1.045, p  ≤  .305).

Discussion

Summary of Findings

In this study, experimental participants had 23% fewer arrests, 14% fewer overall convictions, and 37% fewer jail days in the study period than in the baseline. Control participants had 8% fewer arrests (NS) and 25% fewer jail days in the study period than in the baseline; but overall convictions increased 20%. Study period felony convictions increased 7% for experimental and 12% for control participants.

Despite reductions from baseline, during the study period the experimental participants did not differ significantly from control participants regarding how soon clients were rearrested (relative risk = .94) or number of arrests (experimental arrests 2.4 per person year vs. 2.6) Experimental participants were no less likely to be convicted during the study period than were control clients (.6 per person year vs. .7 per person year), but the number of convictions (if any) was marginally lower (mean of 2.0 vs. 2.6). There was no study period difference between experimental and control groups regarding felony convictions (.3 per person year vs. .3 per person year). Experimental participants did have significantly fewer incarcerations during the study period (2.2 vs. 2.8), but this was off-set by having significantly longer stays when they were incarcerated (mean of 59 vs. 43 days per incarceration).

Attrition from the experimental program was 41%, overall. This is considerably less than many dual diagnosis programs (Brone et al., 2004; Hellerstein, Rosenthal, & Miner, 1995; Lehman, Herron, Schwartz, & Myers, 1993; Van Stelle, Blumer, & Moberg, 2004) but far more than the 9% in the prototype IDDT programs in New Hampshire (Drake et al., 1998).

Even with this relatively high level of attrition, the experimental participants evidenced a very different service pattern than did control participants. Acute hospital and crisis utilization was significantly higher for control participants, while utilization of outpatient services, including medication, was much greater for experimental participants. Mean total costs were $14,809 for experimental participants and $11,069 for control participants. However, most control group costs were generated by a small group extensively utilizing acute hospital services while most experimental group costs reflected broad utilization of engagement-related services.

The study groups were virtually identical in the percentage having institutional days and, if any, in the number of institutional days. This suggests some support for the hypothesis that for this population institutional involvement is a constant, so if jail days go down hospital days will increase (Belcher, 1988).

While very limited, the one measure we had of experimental client functioning over time—the Multnomah Community Ability Scale ratings—showed virtually no change.

Limitations of the Study

Despite randomization, both baseline jail days and baseline mental health service costs were higher for experimental participants than control participants—raising questions regarding study group comparability.

The consequences we measured—criminal justice involvement, hospitalization and institutionalization, attrition, and receipt of comprehensive on-going care—were for the funding agency, Alameda County, the most critical tests of a dual disorders program. However, a fully informative test of the IDDT model would have to include information on substance use and stages of treatment over time. In addition, having measures of mental health status, housing stability, employment and avoidance of health problems such as HIV for both study groupswould be necessary for a full understanding of treatment effects.

Participants were required to be Alameda County residents in order to be eligible. Yet we suspect that because of their lifestyle some of the study participants may migrate between Alameda County and other San Francisco Bay Area counties. We cannot be certain that this has not created a bias against the experimental clients, because they may be more likely to be stably housed in Alameda county than are the control clients. A total of five persons assigned to the experimental group and six assigned to the control group had neither law enforcement nor mental health contact after their index jail release. We do not know if these clients just did very well or if they left the area.

Experimental study participants have no more than an 8–32 month exposure to treatment. Some studies indicate that for many clients three years is necessary in order to achieve abstinence from alcohol and drugs and to stabilize the psychiatric disability (Drake et al., 2001), although differences were apparent in 9 months in the randomized Barrowclough experiment (Barrowclough et al., 2001). However, we have also presented evidence that outcomes for experimental participants did not improve or even declined over time: the attrition rate for those with over two years in the program was 53% compared to the 29% among those with two years or less in the program; and, for the 47 experimental participants with multiple Multnomah ratings there was no improvement in functioning over time.

Causal Ambiguity

The large criminal justice effect sizes found here—reductions of arrest, conviction and jail days by as much as 37% from baseline—are at the high end of other criminal justice reduction study results (Aos, Phipps, Barnoski, & Lieb, 2001), so it seems reasonable to reject the hypothesis that these reductions are simply a regression to the mean. However, the reductions may reflect more than the effects of the intervention. Another Alameda County jail was closed for a period of time during the study period, putting great pressure on the Santa Rita Jail, the focus of our study. Arresting and booking officers and judges may have decided to ameliorate the crowding by informally reducing arrests and misdemeanor (but not felony) convictions and jail days for well-known misdemeanants with substance abuse and mental health problems. These two forces—treatment and overcrowding—may have both been active. With the data available, however, it is not possible to decide the relative strength of treatment vs. overcrowding or whether they differentially affected the study groups. Another implication of reductions due to jail crowding is that it may have limited the treatment effect we found.

Service and Research Implications

Increased control group high end services

High-end treatment utilization increased substantially during the study period for control participants. One explanatory hypothesis is that receiving high quality, caring services in the In-Custody unit in jail made service recipients much more likely to seek treatment after they were released from jail. Only in the experimental program, however, were these services provided in an appropriate mix of engagement-related and high-end services. Increased use of mental health services, especially hospitalization was found in a recent multi-site diversion project (Broner, Lattimore, Cowell & Schlenger, 2004). From a policy perspective, then, services in jail or a diversion program should be paired with an outreach-oriented community program with a long-term focus in order to ensure appropriate use of mental health services.

The IDDT Model

The IDDT model has been promoted as evidence-based (Administration, 2005; Drake et al., 2001; Mercer-McFadden et al., 1998; Mueser et al., 2003; Torrey et al., 2002), but this is the first randomized controlled test of the IDDT model published in which a high degree of fidelity, using the SAMHSA fidelity scale, was documented. There are clearly limitations to this study which in themselves would be enough to account for the fact that our findings are positive in some regards but fairly limited. However, there are other considerations which might also have bearing.

First of all, as noted earlier, there is little evidence from other randomized controlled trials that integrated dual disorders treatment is effective (Donald et al., 2005; Ley et al., 2000). The Alameda County results may also indicate the IDDT model may be less effective beyond its primary evidence base. The most important evidence for the IDDT model represented in the fidelity scale is from a study conducted in seven ACT and intensive case management programs in New Hampshire in the early 1990s (Clark et al., 1998; Drake et al., 1998). Participants in the current study differed substantially from the New Hampshire study participants in having a much higher percentage with criminal justice histories, in coming from an urban vs. rural community, in race/ethnicity (only 21% of experimental participants were white vs. 96% in New Hampshire), and in having a higher proportion of persons dependent on cocaine, methamphetamine, heroin and other street drugs (Clark et al., 1999; Mueser et al., 2001). Thus, we might expect some limits to the “portability” of the model to this different population and setting.

Since the fidelity scale as a whole is as yet not validated, there is also uncertainty as to what elements of the scale correlate highly with good outcomes. Despite overall scores averaging a four on a five point scale, there may have been essential components to the model that were implemented inadequately. For example, fidelity to both peer support (12 step) and family interventions were rated relatively low—though neither one was tested for predictive value in the sole existing fidelity study (McHugo, Drake, Teague, & Xie, 1999). Or components of the model may need modification for an offender population. For example, a successful ACT-based pilot used staff to patient ratios of 1:6 (McCoy et al., 2004).

More fundamentally, a recent study has shown that even the well-validated fidelity scale for ACT programs may be a relatively poor predictor of program outcomes (Bond & Salyers, 2004). A second recent study used an early version of the ACT fidelity scale to test whether lack of fidelity explained much poorer outcomes in a British ACT team than had been reported in the United States; it did not, as both teams achieved high fidelity (Fiander, Burns, McHugo, & Drake, 2003). Thus, evidence is weak that what is measured by fidelity scores is actually predictive of outcomes in model replications.

The current study serves as a caution that in addition to studying the implementation process and documenting fidelity to the IDDT model—the thrust of current SAMHSA grants—a great deal of research remains to be conducted on the constituent components of effective treatment for co-occurring disorders for different populations. Looking toward the future, implementation research that applies or adapts the IDDT model will be of much greater value if it (a) measures conceptually-guided specifics of the treatment process (such as organizational functioning) that can help refine and validate the fidelity scale; and (b) measures outcomes using randomized controlled trials. While randomized controlled trials are critically needed for IDDT programs in general, there is a particular need for trials with a criminal justice population and—potentially—for trials which test adaptations of the model for this population.