INTRODUCTION

Approximately 50% of those individuals who suffer from severe mental illness also have a co-occurring substance use disorder (Regier et al., 1990). These dual disorder individuals are more prone than other people with severe mental illness to experience a number of other negative outcomes, including higher relapse rates, more frequent hospitalizations, more physical health problems, greater violence, higher incarceration rates, and more homelessness (see Drake et al., 2001a). The presence of both disorders poses difficulties not only for individual consumers, but also for service systems. For example, dual disorder individuals incur significantly more treatment costs than other clients with severe mental illness (Bartels et al., 1993; Dickey & Azeni, 1996). Clinicians often regard dual disorder clients as more resistant to treatment and less compliant with treatment regimens. Given the high prevalence of the co-occurring conditions, the severity of their problems and the expensive cost of care, it is imperative that service systems find effective ways to serve these individuals.

Traditionally, most service systems have followed an approach of “parallel treatment” for persons with dual disorders. With parallel treatment, agencies and individual practitioners specialize in treating either mental illness or substance abuse; few programs offer both mental health and substance abuse treatment within the same treatment program. In practice, however, dual disorder clients are often excluded from one or both systems of care. In those rare instances when dual disorder clients do receive treatment from both systems, treatment interventions are rarely co-ordinated and clients are left to their own devices to assimilate the often disparate messages sent to them by the mental health and substance abuse therapists. Research suggests that the client outcomes of such treatment approaches are generally poor (see Drake et al., 2001a).

Integrated Treatment

Over the past 15 years a movement to create “integrated treatment” models for dual disorder clients has gained momentum. The essential feature of integrated treatment is that the same clinician (or team of clinicians) provides both mental health and substance abuse treatment in a co-ordinated manner. There has been considerable variability and evolution in the design of integrated treatment for dual disorder clients. Recent research and theory suggests integrated treatment should incorporate the following key components: (a) assertive outreach, which is needed to engage many dual disorder individuals into treatment; (b) motivational interventions, which are needed to gradually help individuals who are not committed to abstinence to develop personal goals for substance abuse recovery; (c) a stages-of-treatment approach, which includes the following phases: engagement, persuasion, active treatment, relapse prevention; (d) cognitive behavioral counseling, which helps people develop skills for an abstinent life style; and (e) interventions to strengthen social networks supportive of recovery. Interventions must also take a long-term perspective, be culturally competent and comprehensive. The integrated treatment approach to substance abuse can be combined with various types of mental health services, such as residential programs, case management and assertive community treatment.

Recent policy initiatives have promoted integrated treatment as a preferred treatment approach for dual disorder individuals. For example, the President’s New Freedom Commission on Mental Health (2003) and the Substance Abuse and Mental Health Administration support integrated treatment as one of six recommended evidence-based practices (Drake, Mueser, Brunette, & McHugo, 2004). There is considerable consensus about the importance of integrated treatment for dual disorder individuals. In addition, the research evidence supporting the efficacy of integrated treatment is growing, although still limited (Drake et al., 1998b, 2001b). The research designs of many earlier studies were inadequate. In addition, studies of integrated treatment have used different interventions, employed different outcome measures, and reported different results in various domains. For example, in one of the better designed studies, clients who received integrated treatment decreased their substance use more than clients in standard case management, but there was no difference between the two conditions in terms of psychiatric symptoms (Drake et al., 1998a). There also has been relatively little research on the effectiveness of integrated treatment for specific subpopulations of people with severe mental illness, such as those who are incarcerated or those who are homeless. Although two prior studies (Drake, Yovetich, Bebout, Harris, & McHugo, 1997; Meisler, Blankertz, Santos, & McKay, 1997) suggested that integrated treatment may be an effective treatment approach for dual disorder homeless persons, neither of these studies used an experimental research design.

Some researchers have argued that the ideal way to serve dual disorder individuals is to combine assertive community treatment (ACT) with integrated treatment (Phillips et al., 2001). Considerable research has documented that clients with severe mental illness experience more improvement on many outcome variables with ACT than other treatments (Bond, Drake, Mueser, & Latimer, 2001; Mueser, Bond, Drake & Resnick, 1998). The current project compares the effectiveness of three treatment approaches: (1) integrated treatment plus assertive community treatment (IACT); (2) assertive community treatment only (ACTO); and (3) standard care (Control).

Cost of Treating Dual Disorder Individuals

The current project also conducted a detailed cost comparison of IACT, ACTO, and standard care. Prior research has generally found no difference between ACT and other treatments in terms of the total societal costs incurred in treating individuals with severe mental illness (Essock, Frisman, & Kontos, 1998; Weisbrod, 1983; Wolff, Helminiak, & Diamond, 1995), including those who were homeless at baseline (Lehman et al., 1999; Wolff et al., 1997). However, the distribution of costs is quite different. ACT programs typically have greater direct treatment costs than standard programs, because they have smaller client to staff ratios. On the other hand, ACT programs typically incur less inpatient mental health costs.

Only two studies have compared the costs of integrated treatment vs. other treatments for treating dual disorder individuals. Jerrell (1996) reported that a behavioral skills program had lower mental health costs than either a 12-step program or an intensive case management program. However, this quasi-experimental study did not examine a number of other important categories of costs, such as criminal justice expenses, other health care costs, and maintenance costs, and there were significant differences between groups in costs at baseline. Jerrell also reported that there were significant treatment implementation problems, so that the study results may not generalize to a situation where the treatment models are implemented with greater fidelity. In a randomized study Clark et al. (1998) compared standard case management to integrated treatment. The total costs of the integrated treatment condition were not significantly less than the standard case management program.

Study Hypotheses

Researchers for this study had the following hypotheses: (1) IACT and ACTO clients would show more improvement on days in stable housing than Control clients; (2) IACT and ACTO clients would show more improvement on the psychiatric symptom measures than the Control clients; (3) IACT and ACTO clients would be more satisfied with their treatment program than Control clients; and (4) IACT clients would show more improvement on the substance measures than both ACTO and Control clients.

We expected the total societal costs associated with the IACT condition to be less than the total costs of either the ACTO or Control conditions. We also predicted that the outpatient costs for both the IACT and ACTO groups would be greater than the outpatient costs for the Control clients; on the other hand, we expected the inpatient treatments costs would be greater for the Control clients than the IACT and ACTO clients. We also hypothesized that specialized substance abuse treatment costs of the IACT clients would be less than costs for the ACTO and Control clients, because the IACT program would be providing substance abuse treatment directly.

METHOD

Sample and Research Design

To be eligible for this study individuals had to meet the following criteria: (1) be literally homeless, i.e., currently staying in an emergency shelter, living in an abandoned building, sleeping in a car, or sleeping in a public place such as bus depot, park, etc.; (2) have a severe mental illness operationalized as schizophrenia, atypical psychosis, bipolar disorder, recurrent major depression, schizo affective disorder, or delusional disorder; (3) have a DSM-IV substance use disorder; (4) and not be currently enrolled in an intensive case management program. The Structured Clinical Interview for DSM-IV AXIS I Disorders (SCID) developed by First, Gibbon, Spitzer, and Williams (1996) was used to obtain both the psychiatric and substance use diagnoses.

The 196 eligible participants were randomly assigned to one of the three conditions: (1) IACT, (2) ACTO, or (3) Control. Participants were followed for a period of 24 months. The sample size for this project was reduced to 149, because 29 (19%) participants did not sign a release of information form granting access to service utilization data from other agencies and 18 (12%) participants could not be located for a follow-up interview.

The 149 participants had the following demographic characteristics. 80% of the sample were men. Seventy-three percent of the sample were African Americans, 2% other minorities, and 25% Caucasian. The mean age was 40 years and ranged from 18 to 66. The majority of participants (58%) had at least a high school education and were never married (57%). The sample reported a mean of 12.5 days homeless in previous month. Almost half of the sample (48%) received a current diagnosis of schizophrenia, 19% had schizoaffective disorder, 11% had atypical psychotic disorder, 11% had bipolar disorder, 9% had major depression-recurrent disorder, and 2% had delusional disorder. The average Global Assessment of Functioning score (American Psychological Association, 2000) was 43.66, indicating considerable impairment. All of the participants had one or more substance use disorders; 46% met criteria for a substance dependence disorder for alcohol and/or drugs; 64% met criteria for substance abuse disorder for alcohol and/or drugs. Forty percent had an alcohol-only diagnosis, 18% had a drug-only diagnosis, and 42% had both drug and alcohol disorders. Cocaine (or crack) was the most frequently used drug (34%) followed by cannabis (19%).

Procedures

IRB approval was obtained for the study procedures. Participants were recruited from a variety of settings, including emergency shelters, soup kitchens, psychiatric hospitals, and street locations frequented by homeless people. A research screener first observed and talked briefly with numerous homeless individuals, trying to discern whether an individual might be eligible for the project. Once the screener suspected that a given individual might qualify for the project, the person was invited to be formally screened for project eligibility. Informed consent was obtained for this process and individuals were paid $10 to complete the SCID and answer the other eligibility questions. The 258 individuals who were not eligible for the project were referred to other agencies for services. The most frequent reason for lack of eligibility was failure to meet the severe mental illness criterion (70% of the non-eligibles).

Eligible participants were introduced to a research interviewer who obtained informed consent for the treatment study and completed a baseline interview. The research interviewer then arranged for each study participant to be linked with their assigned treatment condition. Often the research interviewer transported participants to the location of the assigned treatment program. Participants were interviewed monthly for 24 months, although not every variable was measured monthly. Participants were paid $5 for the shorter interviews which took less than 30 min. Participants were paid $10 for the longer quarterly interviews which usually lasted for about an hour. Interviews were conducted at a location convenient for the participants, including the research office, treatment agencies, emergency shelters, and the residences of participants. With one exception, all interviewers had master’s degrees in social work, counseling, gerontology, or clinical psychology; one interviewer only had a bachelor’s degree in social work, but she had 15 years experience working with homeless people with mental illnesses.

Treatment Conditions

The IACT condition was implemented at a small community mental health agency. A new IACT clinical team was created for this project, although many of the staff had prior experience providing ACT to homeless people with severe mental illness. The ACTO condition was implemented in two other mental health agencies in the community. One of the agencies had a psychosocial rehabilitation day treatment on site which was also used by some of the study participants. The other agency operated its own transitional housing facility which was used by some of its participants. Both the IACT and ACTO teams received training and follow-up consultation from project personnel regarding ACT treatment principles and practices. Additionally, research personnel and national experts provided the IACT team with training and consultation on integrated treatment principles and services. The IACT had a substance abuse specialist on staff and provided substance abuse services directly as part of the ACT team. These services included individual substance abuse counseling and bi-weekly treatment groups. The ACTO team was instructed to refer clients to other community providers for outpatient or individual substance abuse services and to 12-step groups.

Participants assigned to the standard care control condition were shown a list of community agencies that provided mental health and substance abuse treatment. Research staff also provided these participants with current information about openings at the various agencies and provided linkage assistance to help participants access services at these agencies.

Treatment Implementation and Outcome Measures

Treatment Implementation

Every month participants reported on how many days that they had seen a staff member of their assigned treatment program. In addition, participants indicated how many days during the past month that a staff member of the assigned program had talked with them about their substance use. Research staff also assessed the treatment fidelity of the conditions to ideal standards of assertive community treatment, using a revised version of the Dartmouth Assertive Community Treatment Scale (DACTS) developed by Teague, Bond, and Drake (1998). Based on monthly observations of team meetings and interviews with clinical staff, research staff rated the IACT and ACTO programs on the DACTS at two points in time, 12 and 24 months after project initiation. The DACTS used in this study consisted of 26 items measuring the human resources, organizational characteristics, and service operations of the IACT and ACTO teams (Winter & Calsyn, 2000). Each item was rated on a 5-point scale, where high scores indicated an ideal level of fidelity with the ACT model. The research staff also evaluated the IACT and ACTO programs using a similar 5-point scale on nine pilot items designed to assess the integrated treatment philosophy (e.g. use of motivational interviewing, comprehensive substance abuse assessment, and a skills-approach to substance abuse counseling).

Client Satisfaction

Every 3 months participants reported their satisfaction with the assigned treatment program using a 10-item scale developed for this project. Items from existing scales were modified to be more appropriate for the target population and treatment settings. For each item participants indicated their degree of satisfaction on a 6-point scale, with higher scores indicating greater satisfaction. Alphas for this scale ranged from .85 to .89. Participants were only asked to complete the client satisfaction scale if they had seen their assigned program within the past 3 months. Twenty-two percent of the participants in the IACT group, 19% in the ACTO group, and 76% in the Control group had at least one missing value on this variable. Missing values were replaced with the mean client satisfaction score from available time points for that participant. Two participants in the Control group who had missing values on all 8-time points were eliminated from this analysis.

Stable Housing

Every month participants reported on their housing situation. Days living in stable housing (i.e., living in one’s own apartment or a boarding home) were used in this study.

Mental Health

The 24 item Brief Psychiatric Rating Scale (BPRS) was used to assess psychiatric symptoms (Lukoff, Nuechterlein, & Ventura, 1986). The BPRS has been widely used to assess psychiatric symptoms in patients suffering from severe mental illness. For each item, scores can range from 1 = “no symptoms” to 7 = “extremely severe”. The average item score was used in this analysis.

Substance Use

Participants were asked to report the number of days in the past 90 days that they had used alcohol and also the number of days they used other substances. Because a significant number of participants only used alcohol, there were many “zero” values on days used other substances. Therefore, to eliminate skewness, we created a “highest days used” variable for each participant based on which substance (alcohol or drugs) they used most frequently. In addition, every 3 months the research interviewer assessed the severity of both alcohol and drug use with two 5-point scales that have been used in many previous studies (Carey, Coco, & Simmons, 1996; Drake, Osher, & Wallach, 1989). Higher scores indicated greater severity with 1 = client has not used alcohol (or drugs), and 5 = meets criteria for severe use plus related problems are so severe that make non-institutional living difficult. Again, to eliminate skewness in the data we created a “highest substance abuse” rating for each participant, depending on which substance (alcohol or drugs) the participant abused the most.

Measurement of Service Use and Cost Variables

This study classified costs into the following major categories: Outpatient Costs, Inpatient Costs, Emergency Shelter Costs, and Total Costs. Outpatient Costs had the following sub-categories: Direct Treatment for the IACT and ACTO conditions, Other Mental Health, Other Substance Abuse Treatment, Physical Health Care, and Psychosocial Rehabilitation Center. Inpatient Costs had the following sub-categories: Mental Health, Substance Abuse, and Physical Health Care.

Detailed service utilization and cost data, along with data on income maintenance benefits, were collected for each study participant for 24 months following random assignment and for the 6-month period immediately preceding assignment. Service use data were obtained from service agencies, claims records, and participant self-report. Services provided by the intervention agencies were valued at their average resource costs, which were based on a full cost accounting of the resources used to produce the services. Payment rates were used to value other health and mental health services. Total costs are expressed in 2001 dollars. The data collection and costing methods used here are consistent with other published studies (Clark et al., 1998; Wolff et al., 1997; Wolff, Helminiak, & Tebes, 1997).

The unit costs of the intervention agencies were based on their fiscal data, which was augmented to reflect the fair market rate for owned space and the value of time contributed by volunteers (valued at the minimum wage). Services provided by the Department of Mental Health (DMH) were valued in terms of their current year costs; and DMH contracted services were valued in terms of their current year payment amounts to the providers of the services. Utilization of other health and mental health services was determined from the management information systems of state and local facilities. Medicaid claims data were used to identify services provided to these clients by facilities located in and around St. Louis. To ensure completeness, these records were verified against self-report data and administrative records received from five hospitals located in and around St. Louis. Mean Medicaid payment rates were used to value health and mental health services provided by these other agencies. Utilization of emergency shelter services was based on participant self-report information and priced according to an inflation-adjusted rate used by Culhane, Metraux, and Hadley (2002).

In addition to treatment costs, this study also estimated transfer payments and other maintenance benefits. The Maintenance Cost categories were Social Security (FICA), Social Security Disability Insurance (SSDI), Supplemental Security Income (SSI), and income maintenance benefits (e.g., food stamps, TANF, General Relief). Maintenance costs were also valued at their current year payment amounts.

Data Analyses

A 3  ×  5 (or 3  ×  4 if no baseline assessment) ANOVA design was used to analyze the dependent variables. Treatment condition (IACT, ACTO, and Control) was a between-groups factor and Time (baseline, 1–6 months, 7–12 months, 13–18 months, and 19–24 months) was a within-groups factor. Post-hoc analyses were done using the Tukey’s HSD procedure with significance levels set at p  < .05. Variables that were measured more frequently than every 6 months were aggregated using an average function. There was no baseline assessment for program contacts, substance abuse contacts, or client satisfaction. We also used the Huynh-Feldt correction for the time main effect and treatment condition by time interaction since they all violated the sphericity assumption.

Power estimates were calculated using the software program G*Power (Erdfedler, Faul, & Buchner, 1996). For the purposes of these calculations, it was estimated that there were three levels of the between-subjects factor and five levels of the within-subjects factor, although each dependent variable is not always examined at five time periods. The sample size for these calculations was determined by averaging the sample sizes that occur across the study dependent variables, which range from 139 to 149 (m = 147). The population correlation between the individual levels of the repeated measures factor (ρ) was estimated by averaging the observed correlations between the study dependent variables of interest (ρ = .64). Assuming a potential moderate effect size (using Cohen’s convention of .25) and an alpha level of .05, a sample size of 147 produces 90% power to detect a between-subjects effect and 100% power to detect a within-subjects effect and interaction of between-subjects and within-subjects effect.

RESULTS

Attrition

As indicated earlier only 149 of the original 196 participants provided data for this cost study. Statistical analyses of demographic, diagnostic and outcome variables at baseline were made to determine if the participants in this study were different from the 47 participants who did not provide data for this study. One-way ANOVA’s and chi-square analyses indicated that the two samples differed at the p  < .05 level of significance on only two of 24 variables. The final study sample reported significantly fewer days of alcohol use and significantly more days of stable housing at baseline than individuals who did not provide service utilization data. The researchers also made treatment group comparisons on the baseline assessment of the five outcome variables for the final sample of 149. There were no significant treatment group differences (p  < .05) on four of the outcome variables. However, there was a significant difference on the BPRS, F(2, 146) = 3.833, p  = .02, η2 = .07. The IACT clients had a significantly (p  < .05) lower mean on the baseline BPRS than both ACTO or control clients.

Treatment Fidelity and Treatment Diffusion

The average DACTS scores for the IACT and ACTO teams were moderately high. In year 1, the average item score was 3.85 for the IACT condition and 3.67 for the ACTO condition. In year 2, the average item scores were 3.85 for the IACT condition and 3.64 for the ACTO condition. Fidelity data suggested some treatment diffusion occurred. For example, on the DACTS item that measured the intensity of individualized substance abuse services provided by the team to clients, the IACT team scored in a high range (average 4.4 for the 2 years) but the ACTO teams scored in a moderate range (3.16), indicating that the ACTO teams provided some substance abuse treatment to their clients. Key informant reports indicated that during the course of the project the ACTO programs obtained some training on integrated treatment.

Treatment Implementation

Program Contact

Table 1 displays the program contact data for the three conditions over the four time periods. There was a significant main effect of treatment condition on the program contact variable, F(2, 143) = 40.87, p  < .001, η2 = .36. Post-hoc analyses indicated that the ACTO condition had significantly more contact with their clients than both the IACT and Control conditions, and the IACT program had significantly more contact with their clients than the control clients. There was no main effect of time, F(3, 429) = .45, p  = .68, η2 = .01, nor treatment condition by time interaction, F(6, 429) = .87, p  = .50, η2 = .01. Although we had hypothesized that the ACTO and IACT programs would have more contact with their clients than the control condition, we had not expected the ACTO condition to have more contact with their clients than the IACT program.

TABLE 1 Average Monthly Contacts by Time and Condition: Mean and Standard Deviations
Full size table

There was also a main effect of treatment condition in terms of substance abuse treatment contacts from the assigned program, F(2, 143) = 9.60, p  < .001, η2 = .12. Post-hoc analyses indicated that the clients in the ACTO and IACT conditions had more substance abuse treatment contacts with their assigned program than Control clients. However, there was no difference between the ACTO and IACT conditions in terms of substance abuse treatment. We had expected the IACT clients to report more substance abuse treatment with their case managers than ACTO clients. There was no main effect of time, F(3, 386) = .69, p  = .56, η2 = .01, nor treatment condition by time interaction, F(5, 386) = .72, p  = .64, η2 = .02.

Outcomes

Client Satisfaction

Table 2 displays the means and standard deviations for the outcome variables of the three treatment conditions across the four measurement periods. There was a significant main effect of treatment on client satisfaction, F(2, 144) = 3.46, p  = .03, η2 = .05. Post-hoc analyses indicated that clients in the ACTO and IACT conditions were significantly more satisfied with their treatment than clients in the Control condition. There was no significant difference in client satisfaction between the IACT and ACTO conditions. There was no main effect of time, F(3, 410) = 1.17, p  = .32, η2 = .01, nor treatment condition by time interaction, F(6, 410) = 1.70, p  = .12, η2 = .02.

TABLE 2 Means and Standard Deviations for Outcome Variables by Time and Condition
Full size table

Stable Housing

The main effect of treatment on stable housing was statistically significant, F(2, 145) = 3.76, p = .03, η2 = .05. Post-hoc analyses indicated that clients in both the ACTO and IACT conditions had significantly more days in stable housing than Control clients. There was no significant difference between the IACT and ACTO clients in terms of days in stable housing. There was also a main effect of time on stable housing, F(3, 440) = 66.20, p  < .001, η2 = .31. In general, over time the clients increased the numbers of days in stable housing. Post-hoc analyses indicated that all of the increases between successive time periods were statistically significant, except the last two time periods. There was no significant treatment by time interaction, F(6,440) = 1.93, p  = .07, η2 = .03.

Mental Health

There was no significant effect of treatment condition on the BPRS scale, F(2, 139) = 2.34, p  = .10, η2 = .03. There was a main effect of time, F(3, 438) = 41.72, p  < .001, η2 = .23, indicating that client symptoms improved over time. Post-hoc analyses indicated that most of the improvement occurred during the first 6 months of treatment. There was no treatment by time interaction, F(6, 438) = 1.12, p  = .35, η2 = .02.

Substance Use

There was no main effect of treatment condition on the interviewer rating of substance use, F(2, 136) = .32, p  = .72, η2 = .01. There was a main effect of time, F(4, 516) = 18.35, p  < .001, η2 = .12; substance abuse severity ratings decreased over time. There was no treatment condition by time interaction, F(8, 516) = .69, p  = .69, η2 = .01. On the days of substance use variable, there was no main effect of treatment, F(2, 144) = .63, p  = .53, η2 = .01. There was a main effect of time, F(4, 506) = 3.95, p  < .01, η2 = .03; over time participants reported using substances less often. Post-hoc analyses indicated that most of the reduction in substance use occurred in the first 6 months of treatment. There was no treatment by time interaction, F(7, 506) = .42, p  = .89, η2 = .01.

We also did a comparison of the differences between the two agencies that implemented the ACTO condition. There were no significant differences between the two agencies on any of the outcome variables. However, there was a significant difference between the two agencies on the program contacts variable. The agency that operated a psychosocial rehabilitation day treatment program had significantly (p  < .05) more contact with their clients than the agency that operated its own transitional housing facility.

Costs

Table 3 displays the means and standard deviations for the cost variables for each treatment condition for the five measuring periods. One of the most striking patterns in the data is the large standard deviations for most of the cost categories. Such large standard deviations indicate that some people had low costs, while other individuals had substantial costs. In fact, for any one time period many individuals incurred no costs for the more specific categories, such as physical health costs. Therefore, using parametric statistics to compare group differences on specific cost categories was not appropriate. Thus, statistical comparisons were only made for the major cost categories (i.e., outpatient, inpatient, emergency shelter, total, and maintenance).

TABLE 3 Costs in Dollars per 6-Month Period By Treatment Condition
Full size table

Post-Treatment and Outpatient Costs

There was a significant main effect of treatment condition for outpatient treatment costs, F(2, 146) = 12.47, p  < .001, η2 = .15. Post-hoc analyses indicated that the IACT and Control program cost significantly less than ACTO program, but there was no significant difference in outpatient treatment costs between IACT and Control. There was also a significant effect of time, F(3, 374) = 39.64, p  < .001, η2 = .21. Post-hoc analyses indicated that the outpatient treatment costs were significantly greater after the study began than in the prior 6 months. There was an interaction between treatment condition and time, F(5, 374) = 6.10, p  < .001, η2 = .08, such that the ACTO condition increased their outpatient costs at a faster rate than the other two conditions.

As expected, direct treatment costs represented the largest share of the outpatient costs for both the IACT and ACTO conditions (see Table 3). “Other outpatient mental health treatment” was the largest outpatient cost category for the Control condition. Clients in the IACT condition incurred less outpatient substance abuse costs than the other conditions as expected. As Table 3 indicates, outpatient mental health costs from other agencies were also lowest in the IACT condition.

Inpatient Costs

There was no main effect of treatment condition for inpatient costs, F(2, 146) = .10, p  = .90, η2 = .01. Also, there was no main effect of time, F(4, 512) = 1.58, p  = .19, η2 = .01; nor was there any interaction between treatment condition and time, F(7, 512) = .45, p  = .87, η2 = .01. Inspection of Table 3 indicates that mental health treatment represented the largest share of the inpatient treatment costs, followed by substance abuse treatment, and then physical health care. It is also important to point out that the standard deviations for inpatient treatment costs were quite large; many individuals incurred no inpatient treatment costs, while a few individuals had very lengthy and expensive hospitalizations. For example, during months 1–6 of the study, 39% of the sample incurred no inpatient costs; and during months 19–24, 60% of the sample had no inpatient costs.

Emergency Shelter Costs

There was no main effect of treatment condition on emergency shelter costs, F(2, 146) = .26, p  = .77, η2 = .01. However, there was a main effect of time, F(3, 458) = 10.04, p  < .001, η2 = .06. Shelter costs decreased over time. Post-hoc analyses indicated that shelter costs increased in the first 6 months of treatment and then decreased to levels lower than the pre-treatment period. The treatment condition by time interaction was not statistically significant, F(6, 458) = .85, p  = .54, η2 = .01.

Total Costs

There was a main effect of treatment condition on total costs, F(2, 146) = 4.00, p  = .02, η2 = .05. Post-hoc comparisons indicated that the IACT and Control conditions had significantly lower total costs than the ACTO condition, but there was no significant difference in total costs between IACT and Control. A careful inspection of Table 3 indicates that greater outpatient treatment costs (direct treatment costs, other outpatient mental health costs, and substance abuse treatment costs by other agencies) incurred by the ACTO condition is primarily responsible for the significant difference between treatment conditions in terms of total costs. There was a main effect of time, F(3, 501) = 3.19, p  = .02, η2 = .02. Post-hoc analyses indicated that costs increased significantly in the first 6 months of the study and then leveled off. There was no significant treatment condition by time interaction, F(7, 501) = 1.25, p  = .28, η2 = .02.

Maintenance Payments

Table 4 displays the maintenance payments that participants received over the course of the study. There was no main effect of treatment condition on total maintenance payments received, F(2, 146) = .51, p  = .60, η2 = .01. There was, however, a main effect of time, F(3, 440) = 9.69, p  < .001, η2 = .06. On average, the amount of maintenance payments that participants received increased over time. Post-hoc analyses indicated that the amounts received between the first two time periods were not significantly different from each other; nor were the amounts received during the last two time periods significantly different from each other. However, the maintenance payments received at non-adjacent study periods was significantly different from each other. There was no treatment condition by time interaction, F(6, 440) = .47, p  = .89, η2 = .01.

TABLE 4 Average Maintenance Cost in Dollars Per 6 Month Interval and Treatment Condition
Full size table

We also analyzed the cost data with non-parametric statistics. The results were virtually identical to the ANCOVA analysis. The only difference concerned the treatment group effect on total outpatient costs. Post-hoc comparisons using ANCOVA found that outpatient costs were significantly greater for ACTO than IACT and Control, but there was no difference between IACT and Control. Post-hoc comparisons using the non-parametric Mann–Whitney test also found that ACTO costs were significantly greater than IACT and Control. In addition, IACT costs were significantly greater than Control costs in the Mann–Whitney analysis.

DISCUSSION

Client Outcomes

The results provide partial, modest support for the study’s hypotheses concerning the effectiveness of IACT and ACTO on client outcomes. As predicted clients in both the IACT and ACTO conditions had better housing and consumer satisfaction outcomes than did Control clients who received standard care, replicating previous research comparing ACT with other treatments (Bond et al., 2001). These findings are important in light of the plethora of problems experienced by dual disorder individuals, and because other treatment studies typically show poor outcomes for homeless people with dual disorders (see Morse, 1998).

Contrary to our prediction, the IACT and ACTO conditions did not produce better outcomes in terms of psychiatric symptoms. Drake et al. (1998a) also found that integrated treatment was no more effective than standard case management in reducing psychiatric symptoms. Past research comparing ACT with standard care has produced mixed results in terms of psychiatric symptoms; some studies report superior outcomes for ACT, whereas other studies report no differences (Bond et al., 2001).

Also, contrary to our prediction, IACT did not reduce substance use more than ACTO or Control. The treatment fidelity and treatment diffusion problems noted earlier probably contributed to the absence of significant findings in this area. A prior study had reported that the amount of improvement in the substance abuse domain was highly correlated with the fidelity of the integrated treatment program (McHugo, Drake, Teague, & Xie, 1999). However, it is important to note that many clients, regardless of treatment condition, reduced their substance use as evidenced by the significant time effects on both the self-report of days used substances variable and the interviewer rating of substance abuse severity.

Cost Findings

The results also partially supported the cost predictions. As expected, IACT participants had the lowest substance abuse treatment costs from other outpatient agencies. As expected, ACTO did have outpatient costs higher than standard community services; surprisingly, however, the outpatient costs for IACT and standard community services were comparable. Contrary to expectations, the differences in inpatient costs across the three treatment conditions were not statistically different. Prior cost studies have generally reported lower inpatient costs for ACT vs. other treatments. At least four possible explanations can be offered for our discrepant results. First, the IACT and ACTO programs in this study did not control admissions and discharges to inpatient facilities. Second, Missouri, like most other states, has undertaken a number of procedures to reduce inpatient mental health costs in the past decade. Therefore, current ACT programs probably have less impact on reducing inpatient costs than earlier ACT programs (e.g., Weisbrod, 1983). Third, large variances and relatively small sample sizes make it harder to detect between group differences. Finally, recent studies in the United Kingdom also found that ACT programs are no more effective than other interventions in reducing hospital costs (Holloway & Carson, 1998; UK 700 Group, 1999).

The cost findings reported here are partially consistent with results of the Clark et al. (1998) study. Comparing integrated treatment to standard case management for a dual disorder sample, Clark and colleagues found no significant cost differences between integrated treatment and standard care in terms of total societal cost or any service component. Our study also found no cost difference between IACT and standard care. The cost results concerning IACT and the Control condition are not surprising. Control client costs are as high as IACT because dual disordered clients incurred considerable costs through a variety of outpatient and inpatient providers; thus it was not less expensive to rely on standard community services for this population.

More puzzling was the finding in the present study that the costs for IACT clients were significantly less than for ACTO clients. As reported, IACT and ACTO showed major differences in outpatient costs in this study. ACTO had greater direct treatment costs due to greater overhead costs. ACTO also had higher costs than IACT for other community provider outpatient mental health costs because they referred many clients for psychiatrist consultation rather than utilizing their own team psychiatrist as recommended by the ACT model. Thus, some of the cost differences observed in this study are undoubtedly due to local conditions and less than desired treatment fidelity to the ACT model.

It is also important to remember that the present study, like the Clark et al. investigation, involved a small sample size (less than 100 per treatment group) and non-normal cost distributions. Extreme variation in costs for a few individuals in relatively small samples adds instability to aggregate statistics and to trend statistics. This type of instability is characteristic of real world community programs, however. A few individuals with extreme behavioral problems can skew the average costs for the program. Thus, one needs to be cautious in generalizing our cost results to other IACT or ACTO programs. Further studies to replicate these findings are needed.

Study Limitations

This study’s conclusions should be interpreted somewhat cautiously, because of treatment implementation problems, treatment drift, attrition and concerns about generalizability to other samples. Although the IACT team received moderately high fidelity ratings for integrated treatment, there were no significant differences between the IACT and ACTO conditions in terms of the self-report data from consumers on the frequency of substance abuse contacts with their assigned program. Further, the absolute frequency of these substance abuse interventions in the IACT condition (as indicated by self-report) was low. Thus, treatment fidelity may have been insufficient to produce the desired changes in substance use. Also, key informant reports indicated that some treatment diffusion occurred, with the ACTO teams providing some substance abuse treatment directly. The net effect of these two problems would be to reduce treatment group differences. Similar implementation and diffusion problems have been reported in other studies on integrated treatment (see Drake et al., 1998).

Sample attrition was another potential methodological problem; 22% of the original sample was excluded from these analyses because they did not authorize the release of the service utilization data necessary for the cost analyses. However, given that there were few significant baseline differences between the sample used in this study and those individuals who did not provide service utilization data, attrition probably did not affect the internal validity of the study but may have reduced statistical power. It should also be noted that clients in the control condition also had fewer assessments on the client satisfaction measure, because many control clients were not in treatment at one or more of the assessment points. Nevertheless, we think that there is little doubt that clients in the IACT and ACTO groups were truly more satisfied with their treatment than clients in the control conditions, many of whom dropped out of control treatment conditions.

Future researchers should consider including the Substance Abuse Treatment Scale (SATS), which is calibrated more finely than the rating scales used in this study. The SATS has detected client change in other treatment studies of dual disorder individuals (McHugo, Drake, Burton, & Ackerson, 1995). Also, given difficulties in subject recall, especially for persons impaired by severe mental illness and substance abuse, researchers may want to use the Timeline Follow Back (TLFB) procedure (Sobell, Maiston, & Sobell, 1980) for collecting self-report information on substance use. These additional measures might improve the ability of future researchers to detect treatment differences on substance abuse.

In addition, the costs reported here are specific to St. Louis and to this sample of study participants. To the extent that resource costs and production methods vary among communities and payment rates vary by state, the absolute and relative costs reported for St. Louis may not generalize to other parts of the country. But it is unlikely that any one community will produce a representative set of findings because local conditions vary particularly by community and in ways that are not systematic. As with all cost studies, this study provides evidence on the absolute and relative differences among particular interventions located in a particular community. By using consistent methods to measure effects and costs, researchers can compare findings across studies to determine which effects appear robust and independent of differences in conditions and samples.

CONCLUSION AND RECOMMENDATIONS

Despite its limitations, this study was methodologically superior to most previous research evaluating the effectiveness of treatment programs for dual disorder individuals. It employed a randomized design, had minimal attrition and followed clients for 2 years. The findings suggest that both assertive community treatment and integrated treatment combined with assertive community treatment show modest promise for improving the lives of homeless people with co-occurring severe mental illness and substance abuse disorders. Further, the study provides some data which indicate that integrated treatment combined with assertive community treatment probably costs no more than standard care, yet provides better outcomes for dual disorder clients. However, because of the implementation problems described earlier, this study did not provide a definitive test of the merits of integrating substance abuse services as part of assertive community treatment.

More research is also needed on the most cost-effective ways to serve people with dual disorders. The length of follow-up should also be increased. Clinical experience and theoretical perspectives suggest that recovery from substance abuse for dually diagnosed persons is a longitudinal perspective that often takes years (Drake et al., 2001a; Test, Wallisch, Allness, & Ripp, 1989). Finally, in addition to studies on outcomes and costs, descriptive and process evaluation research is also warranted. In particular, more research is needed to identify the client characteristics, service ingredients, and environmental factors that predict recovery for dual disorder clients.

Integrated treatment programs can also be enhanced in several ways. Integrated teams should employ more full-time substance abuse counselors, particularly if the caseload contains mostly dual disorder clients. Since the initiation of this project, the practice of integrated treatment has continued to evolve, and teams should incorporate the latest treatment technology, such as that emerging from the evidence-based project on dual disorders (Brunette, Drake, & Lynde, 2002). Teams should frequently monitor the fidelity of integrated treatment programs, using recently expanded and refined measures (Brunette et al., 2002; Mueser, Noordsy, Drake, & Fox, 2003); researchers and program administrators should quickly respond to treatment implementation problems in order to assure the highest levels of fidelity. Researchers and program developers should also consider combining integrated treatment with other intervention strategies that may produce quicker and higher rates of recovery. In particular, residential and supportive housing arrangements (Bassuk, 2003; Brunette, Drake, Woods, & Hartnett, 2001; Brunette, Mueser, & Drake, 2004), certain psychotropic medications (see Drake et al., 2001a), contingency management, and community reinforcement approach (Smith & Myers, 1995) appear to be promising partners for inclusion with integrated treatment. Combining various treatment approaches may lead to even more effective and less costly services for this vulnerable population of people with severe mental illness and co-occurring substance abuse disorders.