Introduction

Operationalizing treatment efficacy has become essential in the field of psychotherapy. Managed health care now requires psychotherapy to produce measurable outcomes and define success concretely. This requirement has resulted in research attempting to identify empirically supported and evidence-based treatments. For example, Butler et al. (2009) represents one such attempt to examine the efficacy of one residential treatment model. However, defining successful residential treatment is often problematic. For example, the medical model traditionally looks for symptom reduction based on diagnostic categories (Mordock 1979), such as those represented in the DSM-IV TR. These symptom reductions can be specific, measurable and uniform. By contrast, social service agencies consider individual and family levels of adaptive functioning as important variants of outcome (Mordock 1979).

Troubled youth are a particularly vulnerable population and treating their mental health concerns can be exceptionally challenging for mental health providers (Alexandre 2008). Most adolescents treated through outpatient services drop out or attend very few sessions (Harpaz-Rotem et al. 2004). As a result, families are increasingly utilizing residential facilities and inpatient psychiatric units to treat their troubled teens (Connor et al. 2002). Therefore, it is important to evaluate the effectiveness of these treatment approaches.

These two milieus are comparable and face similar criticism. Similar to inpatient psychiatric units, residential treatment is an inpatient, or out-of-home, 24-h care facility. Staff make use of the daily living environment in order to facilitate the mental health treatment of the adolescent clients, helping them to build skills and obtain internal stability (Connor et al. 2002; Hair 2005). However, they are typically less restrictive than inpatient psychiatric units and are not licensed as hospitals (Connor et al. 2002; Hair 2005; Larzelere et al. 2001). However, the stay at a residential facility is generally longer than that of a psychiatric inpatient unit (Burnset al. 1999; Hair 2005; Larzelere et al. 2001; Lyons et al. 1998). Similar to inpatient psychiatric units, adolescents in residential treatment are generally experiencing extreme behavioral and psychological disruptions in their lives and have been unsuccessful in outpatient treatment (Blanz and Schmidt 2000; Connor et al. 2002).

The outcome research on adolescent residential and inpatient treatment should be examined together due to the fact that such treatments offer services to virtually the same populations, using virtually the same modalities. These two types of treatment are often distinguishable only by treatment duration. Further, the outcome literature on both types of treatment is scant enough to warrant joint review. Such joint review has some precedence in the literature (Lyman and Campbell 1996).

There are various subtypes of residential treatment programs (Butler and McPherson 2007). Some programs function from an ecological perspective, addressing individual concerns with the adolescent and looking at the interaction between the child and their environments. Others utilize a therapeutic community approach, addressing problem behaviors utilizing peer influence.

While this article focuses primarily on the psychotherapeutic component of these program models, many programs place significant emphasis on other rehabilitative components such as education and family functioning. Although recently research has begun to focus more on the effectiveness of the psychotherapeutic aspect of this type of intervention (Leichtman et al. 2001; Lyons and Schaefer 2000; Lyons et al. 2001; Wilmshurst 2002), it is necessary to consider the role these other components play in fostering change in emotional and behavioral problems. While that focus is outside the scope of this article, it remains an essential component deserving further investigation.

Critics frequently scrutinize inpatient psychiatric units and residential treatment programs, challenging their necessity and effectiveness. Research evaluating treatment outcomes in these settings has become a necessary means of determining the future of these milieu (Hair 2005). Ensuring that adolescents receive the care necessary to maximize healthy development means understanding the benefits that these treatment programs can provide (Lyons et al. 2001).

For decades, critics have expressed cynicism regarding the necessity of residential treatment, questioning the need to remove children from their home environments (Burns et al. 1999; Hair 2005), especially in light of the tremendous cost of such programs (Blanz and Schmidt 2000; Lyons et al. 1998; Lyons et al. 2001). Some question how residential therapy persists when it lacks evidence of effectiveness, suggesting that there is substantial harm in placing children with behavioral problems together (Barth 2005). Others recognize the benefits of such care and advocate for increased regulation, ensuring utilization by only the population it most benefits (Burns et al. 1999) and protecting clients from potentially harmful or abusive situations (“Stop child abuse in residential programs for teens act of 2008” 2008). Still others cite the increased risks of maltreatment which may come to youth in residential or other out-of-home care (Overcamp-Martini and Nutton 2009) and cite the need for legislation which monitors youth residential care more carefully (Nunno 2009).

Opponents of residential treatment cite different problems in the industry. Ward (2004) asserts that adolescents will respond in a variety of ways to the structure of residential care programs. When residential treatment programs attempt to impose structure onto youth that have typically experienced chaotic, abusive or neglectful environments, adolescents may experience such structure as confusing, strange, and uncomfortable (Ward 2004). Underwood et al. (2004) indicate that residential treatment may even be traumatic for adolescents, as many are forcibly placed in residential treatment. They note that separation from familiar places and people may be deeply distressing for some youth. Wilmshurst (2002) asserts that separation from familiar surroundings could exacerbate anxiety in youth with already high anxiety levels. In addition, concerns about the potential of abuse and misuse of disciplinary tactics inspired recent legislative action addressing these concerns, aiming at ensuring client’s safety and rights (“Stop child abuse in residential programs for teens act of 2008” 2008).

Similar concerns plague the psychiatric inpatient service industry. Blanz and Schmidt (2000) discuss the concern with applying qualitatively defined rigid criteria to inpatient psychiatric admissions. These criteria are helpful guidelines to use in clinical practice. However, their disadvantage is that qualitative definitions of appropriateness for this setting are often ambiguous. For example, it may be difficult to determine if the severity of a disorder warrants inpatient treatment. This ambiguity leaves open the possibility of inappropriate admissions. More recently, some inpatient facilities have been using quantitative measures to assist in admissions determination. The CGAS (Shaffer et al. 1983) provides an assessment of the child’s overall functioning within several areas (family, school, peers, daily routines). If the child scores up to 50, inpatient admission is warranted.

A further critique of such rigid criteria is that it also ignores the potential resources available to the client. For example, the availability of some outpatient services or the possibility of residential treatment can influence the appropriateness of an inpatient admission. Further, economic factors may play influence inpatient admittance. Cost rather than need can influence the decision; pressure from insurance companies can directly impact the length of inpatient stay and in turn the effectiveness of this treatment (Blanz and Schmidt 2000).

Are such critiques legitimate? In order to answer this question, we review the outcome literature on adolescent residential and psychiatric inpatient treatment. Our review considers the ways that studies measure and define treatment outcomes, identifying some of the variables affecting outcome measurement. We review how research uses symptom reduction and social functioning as methods of measuring success in adolescent residential and inpatient settings. Further, we consider studies examining theoretical orientation and client characteristics as variable in outcome. We also address methodological issues, including cultural confounds, that exist in the current outcome literature. Finally, our review points to needed directions for future research.

Methods

Computer database searches were used to identify articles for this analysis. In November of 2007, searches were carried out using the EBSCO host search engine. The following databases were included: ERIC, Psych Articles, EBSCO Academic Search Premier, Psych INFO, and Medline. The Search terms used were: adolescents, youth, residential, outcomes, treatment, therapy, methods, philosophy, and modality. Yielded abstracts were prescreened for the following criteria: (a) target population of the study was identified as adolescents between the ages of 12 and 18, (b) adolescent population was treated in residential treatment, (c) the article measured treatment outcomes, (d) the article was published in a peer-reviewed journal (e.g., no dissertations or books), and (e) the article was published in English. In cases where the abstract did not provide information germane to the listed criteria, the article was secured and reviewed for relevancy. This screening method resulted in 13 studies that were secured and reviewed. The studies are reviewed below collectively in terms of elements of the treatment literature which they address.

Results

Measuring Treatment Outcome

How a study defines success in residential treatment determines how outcome is measured. In the context of residential treatment, there are a number of variables affecting measurement of success. Some of these variables are treatment attrition, poor research design, and lack of a control group (Blanz and Schmidt 2000).

In a review of the inpatient treatment outcome literature, Khan (1990) asserts that studies on hospital treatment effectiveness fail to isolate specific healing factors. Khan notes that the absence of these factors, in addition to lack of measurement of influential factors after discharge, may mean that these studies are only measuring the natural course of psychiatric disorders over time. Further, Khan holds that that many outcome studies do not control for variables that interfere with recovery. Examples of such variables are: level of family functioning, peer associations, learning disabilities and inadequate community resources.

Mordock (1979) asserts that defining treatment success in residential treatment centers is a struggle and is often dependent on the defining body. For example, psychiatric accrediting bodies want outcome evaluation based on psychiatric diagnosis and symptom relief. Social service agencies believe evaluations should consider the individual in the context of the family outcomes. They look at how family change influences individuals and their adaptability. School systems want to measure academic achievement and decreased delinquency in evaluating outcomes (Mordock 1979).

In response to these competing views, Mordock (1988) advocates for developmental advancement as the measure of client progress rather than symptom reduction, arguing that “symptom reduction is not an adequate criterion of therapeutic growth” (p. 225). Symptom reduction measures do not assess adolescent developmental growth or account for the lack of reliability in behavioral observation reports. Prentice-Dunn and Lyman (1989) assert, “outcome should be conceptualized and measured multidimensionality and multidirectional; in other words, it should allow for positive and negative changes in a variety of behavioral areas” (p. 248). They believe that measuring outcomes in child treatment can be confounded by maturational issues, the difficulty in defining treatment success or failure, and the possibility of nonlinear change in clients. Blanz and Schmidt (2000) argue that most studies considering treatment outcome provide only short-term follow-up and limited clinical measures, rather than measuring the full range of pertinent outcomes.

Symptom Reduction

Much of the current outcome literature assesses symptom reduction. Lyons et al. (2001) evaluate data from 285 adolescents, from ages 12 to 17, placed in one of eight residential treatment centers. The diagnostic classifications of these adolescents are as follows: “post-traumatic stress disorder (27%), attention-deficit hyperactivity disorder (21%), depressive spectrum disorder (17%), oppositional-defiant disorder (11%), bipolar disorder (7%), psychotic disorder (6%), conduct disorder (2%), and adjustment disorder (1%)” (Lyons et al. 2001, p. 235).

This study considers treatment outcomes by type, diagnosis and treatment center. The assessment instrument used is The Acuity of Psychiatric Illness-Child and Adolescent Version (Lyons, 1998). The authors calculate the trajectory of change for each adolescent as a loglinear function of time since admission (Lyons et al. 2001). Results indicate some reduction in risk behaviors, reliable improvement in depression and psychotic symptoms, and worsening anxious and hyperactive symptoms. Analysis shows no significant differences in outcomes between diagnostic groups. However, the authors conclude that residential treatment may be more effective for adolescents with PTSD and internalizing disorders than for adolescents with ADHD or other behavioral disorders (Lyons et al. 2001).

In another study, Colson et al. (1991) examine the clinical records of 49 male and 30 female adolescents in residential treatment, ages ranging from 10 to 17 with one 6-year-old. Thirty-four patients received personality disorder diagnoses, eight conduct disorders, five major affective disorders, six psychotic disorders, and 16 other childhood or adolescent disorders. Notably, 68% of the sample was previously hospitalized elsewhere.

In addition to patient clinical records, this study evaluates staff responses to study participants in an attempt to find connections between adolescent characteristics, therapeutic alliance and treatment outcome. Because they were described in a previously published article (see Colson et al. 1990), the methods of measurement, an assessment instrument, a set of rating scales assessing treatment difficulty, a study of inter-rater reliability, and factor analyses of scale contents are not detailed in this article. These authors find that participants who staff perceived as difficult had the worst treatment outcomes. The authors also conclude that participants’ angry defiance and resistance to treatment was the greatest obstacle in forming effective therapeutic alliance (Colson et al. 1991).

In both of the studies reviewed above, adolescent behavior is the main criteria in assessing treatment outcome: Lyons et al. (2001) calculate clients’ behavior change over a specific timeframe, while Colson et al. (1991) identifies clients’ behavioral characteristics that residential staff perceive as obstructing treatment success. Both studies identify specific behaviors in an attempt to assess individual change and ascertain treatment barriers. Typical social science research measures behavior changes, as these are observable and measurable. From this perspective, decreases in maladaptive behaviors are indicative of successful treatment outcome.

Social and Familial Functioning

Social service agencies, focusing on the person in their environment, typically measure treatment outcome in terms of social and familial functioning. Studies from this perspective place a great deal of emphasis on the interpersonal nature of treatment (Colson et al. 1991). Researchers determine treatment success by evaluating how family dynamics change in response to treatment and how the individual functions in his social environment.

Hooper et al. (2000) examine 111 adolescents, ages 13–16, in a state-run re-education residential treatment program. About 67% of the studied adolescents were male. All subjects had approximately three psychiatric diagnoses on admission, reflecting most major psychiatric categories. However, the most frequent diagnoses were Conduct Disorder, ADHD, Major Depression and PTSD.

This study uses a single sample cross-sectional design that considered outcomes by contacting participants’ case managers at 6, 12, 18 and 24 months following discharge. Case managers rated client post-treatment legal, academic, and level of care functioning as either “satisfactory” or “unsatisfactory” since discharge. A “satisfactory” legal rating indicates that a child had no new legal activity post-discharge. A “satisfactory” school rating indicates that there was ongoing school participation. A “satisfactory” level of care rating indicates there was no re-hospitalization or additional out-of-home care. Community case managers rated student success between 58 and 78%. Notably, successful students were mostly “younger, female, had higher verbal IQs and reading skills, higher parental ratings of internalized symptoms and had not been discharged from the program for as long as the unsuccessful students” (Hooper et al. 2000, pp. 495–496).

Larzelere et al. (2001) evaluate all available data on the first 43 youth, ages from 6 to 17, treated at a residential treatment center. The sample consisted of 21 boys and 22 girls. About 51% had primary diagnoses of depressive disorders, 21% disruptive disorders, 9% adjustment disorders, 9% PTSD, 7% ADHD, and 2% Schizo-affective Disorder.

Authors used The Child Behavior Checklist (CBCL; Achenbach 1991a) at intake, discharge and follow-up and The Children’s Global Assessment Scale (C-GAS; Shaffer et al. 1998) at intake and discharge only. Participants completed follow-up measures between six and 21 months following discharge. The CBCL provides three scales: internalizing (anxiety, depressive, withdrawal, and somatic), externalizing (aggression, delinquency), and total problems. The C-GAS assesses a subject’s overall level of functioning, and is equivalent to the DSM-IV Global Assessment of Functioning (GAF) score (Larzelere et al. 2001).

In terms of CBCL scores, clients showed improvement on most CBCL scales and these improvements were generally maintained at follow-up. The C-GAS showed similar improvements from intake to discharge. However, researchers did not administer the C-GAS at follow-up. C-GAS mean scores at intake, “[M]ajor impairment in functioning in several areas and unable to function in one of these areas” (Larzelere et al. 2001, p. 181) changed to “[V]ariable functioning with sporadic difficulties or symptoms in several but not all social areas” (p. 181) by discharge.

Hooper et al. (2000) and Larzelere et al. (2001) consider client functioning in various social arenas (legal, academic, familial, etc.). Both studies find that client functioning improved following residential treatment. However, in the Hooper et al. (2000) study, the measurement of client functioning is subjective. The authors define success as individuals functioning at “modestly adaptive levels.” In addition, they assess these levels by case workers’ subjective interpretation. In Larzelere et al. (2001), the C-GAS was administered at intake and discharge only. Therefore, researchers base improved functioning on change within the residential setting, not out in society. This tells us little about how well the adolescent will function once presented with the everyday stressors of normal life.

Many residential settings incorporate family involvement into adolescents’ treatment. This practice is based on the principle that therapeutic growth occurs in a setting that combines peer and family influences, as well as the understanding that the adolescent will likely return to the family environment following treatment (Barth 2005). It is for this reason that social service agencies look at family functioning as a variable in outcome.

In a study examining how family involvement and satisfaction affect outcome, Brinkmeyer et al. (2004) examine 47 adolescent inpatients, ages seven to 17 (56% male), in a child and adolescent psychiatric unit. Thirty-four parents completed research questionnaires: 27 mothers, 5 fathers, and 2 “other” relatives. Nine of the 47 adolescents were missing diagnostic data due to lack of parental consent. Of the remaining 38, 82% had Major Depressive Disorder, 24% PTSD, 18% ADHD, 16% oppositional defiant disorder, 11% psychosis, 5% adjustment disorders, 3% substance abuse, 3% developmental delays, 3% anorexia, and 3% bipolar disorder. In addition, 53% of the children had co-morbid diagnoses.

This study uses the Child Behavior Checklist for 4–18 Year Olds (Achenbach 1991a) to obtain parents’ perception of child’s emotional and behavioral disturbances. Parents completed this questionnaire twice: at discharge and at a 9-month follow-up. Psychiatric residents completed the Family Engagement Questionnaire (Kroll and Green 1997) to assess family engagement, and parents completed the Inpatient Parental Satisfaction Index, a measure designed by these authors for this study, in order to measure consumer satisfaction (Brinkmeyer et al. 2004).

This study finds a relationship between clinicians’ ratings of parent engagement and parent satisfaction with treatment for families whose children had prior inpatient hospitalization. Clinicians rated these families as demonstrating less engagement in treatment, and these parents indicated that they saw treatment as less meaningful and were less satisfied with it.

At the 9-month follow-up, 32% of all study participants had been re-hospitalized and 48% of parents reported either no change or worsening of behaviors, while 52% reported improvement. The 52% that reported improvement demonstrated a larger change in internalizing behavioral problems at the 9-month follow-up. In addition, the decrease in reported internalized behavioral problems correlated with higher ratings of parental engagement and parent satisfaction (Brinkmeyer et al. 2004).

It is important to note that not all the findings from this study reach statistical significance. However, the moderate to large effect size suggests that there is a relationship between therapist’s ratings of parental engagement in treatment, parent satisfaction and client outcome (Brinkmeyer et al. 2004). Further, these findings are consistent with the belief that family involvement has a direct influence on adolescent treatment outcome. It is likely that parent’s positive expectations at the onset of treatment will result in more meaningful participation in the adolescent’s care (Brinkmeyer et al. 2004). It provides further support for the consideration of family functioning and family dynamics as a meaningful indicator of adolescent treatment outcome.

Theoretical Orientation

The various orientations of residential programs become problematic when evaluating the effectiveness of this approach. In fact, some scholars argue that research cannot measure treatment outcome without taking the treatment’s therapeutic approach or theoretical orientation into consideration. The question arises of whether the outcome measure is appraising the quality of the overall effectiveness of residential treatment or assessing the efficacy of the theoretical orientation of the intervention approach.

In a study illustrating this, Mann-Feder (1996) compares outcomes from two theoretically-different residential treatment programs. One residential program is a Therapeutic Community (TC) located within a general psychiatric hospital. It practices a cognitive-behavioral approach focusing on self-regulation, peer confrontation, and insight for behavior change. The other program is part of a psychosocial, non-medical agency. That program’s focus is on behavior modification, using a point system as the basis for a Token Economy (BU). The total sample consisted of 28 adolescents, 14 males and 14 females, between the ages of 14 and 18. This study does not state how many adolescents are from each treatment program. All participants had a diagnosis of Conduct Disorder.

To measure change, Mann-Feder uses the Weschler Intelligence Scale for Children-Revised (Weschsler 1974), the Jesness Inventory (Jesness 1972) which is a personality inventory yielding characteristics predictive of social, emotional, and behavioral problems, and the Conceptual Level Paragraph Completion Test (Hunt et al. 1979) which is a semi-projective instrument measuring self-responsibility and independence. The author also uses the Internal Locus of Control Scale (Levenson 1981), the Tennessee Self-Concept Scale (Fitts 1965), and the Devereux Adolescent Behavior Rating Scale (Spivack et al. 1967), a staff-report measure of client symptoms. Researchers collected data at three points: preceding admission, 3 months after admission, and 6 months after admission.

Statistical analysis found that the TC group differed demographically from the BU group in that they had more males, fewer clients from large families, more clients with a history of out-of home placement, and more clients with exposure to psychotropic medication. Analysis also showed significantly different scores at admission between the two groups. The TC group had less disturbed score on the Devereux Adolescent Behavior Rating Scale’s measures of aggressive dominance of peers, of displayed interest in the opposite sex, and of volatility. In addition, the TC group had poorer scores on the Tennessee Self-Concept Scale’s overall measure of sense of self-worth, a more deviant mean score on the Jesness Inventory Scale’s measure of concern in the interpersonal sphere, and significantly higher full-scale IQs (Mann-Feder 1996).

For both the TC and BU groups, scores moved toward improved functioning, reaching significance on nine of the scales. By the 6 month mark, all clients demonstrated improved maladjustment, decreases in delinquent values and behavior, less egocentricity, more sociability, more ability to control aggression, and greater dissimilarity to clients diagnosed as personality disordered. Further, results showed overall improvement in participants’ self-esteem and personality functioning. Mann-Feder does not find significant changes in scores between or within groups for the Devereux Behavior Rating Scale or the Locus of Control Scale. In regards to differences between the two treatment settings, there were no significant differences in the rate of change in the two treatment groups (Mann-Feder 1996).

The problem with studies looking at therapeutic approach is their lack of clear theoretical definition. Lyman and Campbell (1996) argue that studies often present idealized descriptions of theoretical treatment elements, but fail to offer proof that such elements were actually implemented consistently. Studies of residential treatment programs generally fail to acknowledge whether those working in residential facilities know the theory behind the implemented treatments (Lyman and Campbell 1996). Night and weekend staff not closely supervised may implement program elements in ways that are very different from what theoretically-driven treatment plans specify. Without controlling for or measuring differences in treatment delivery between staff, studies of residential treatment may be ignoring critical elements of outcome (Lyman and Campbell 1996).

Treatment outcome can be difficult to assess due to many factors. Treatment centers can be widely different. For example, variations exist in staff education and qualifications. In addition, treatment organization, site theoretical orientation, and client psychopathology differ significantly between programs. Further, individual and parental participation, family therapy involvement, vocational training components, and post-discharge supports can impact treatment success and are rarely measured (Khan 1990).

Some residential programs have an eclectic or atheoretical approach. Studies of these programs generally focus on the overall residential experience rather than evaluating specific treatment elements. For example, the Larzelere et al. (2001) study discussed earlier investigates the outcomes of a Girls and Boys Town residential facility for adolescents, concluding that this residential treatment model succeeded where other less restrictive treatments for these adolescents had failed. They assert that treatment success may be attributable to program features which include “positively oriented behavioral strategies, including replacement social skills such as calming skills for coping with stressful situations more appropriately” (Larzelere et al. 2001, p. 183). The authors also stress the importance of the treatment environment. They place emphasis on the impact of the constant nature of treatment implementation within this setting. In addition, the authors note the importance of outpatient treatment following discharge. In this sample, 86% of the clients received outpatient treatment following discharge from the program. The authors assert that this element likely contributes to client stability and improved functioning.

Individual Characteristics

Some researchers attempt to identify client qualities predictive of treatment success. Lyons and Schaefer (2000) consider individual strengths of adolescents in residential treatment as variants in outcome. Other studies examine developmental histories (Connor et al. 2002), attributional bias (Curry and Craighead 1990; Dodge et al. 1990), relational issues (Brinkmeyer et al. 2004; Frank et al. 1997; Singer et al. 2000), and internal experience (Blatt et al. 1997) as contributors to outcome. Some of these studies consider individual qualities as influences on treatment outcome, while others look for characteristics of individuals experiencing treatment success.

As discussed earlier, the Hooper et al. (2000) study contacted adolescents’ case managers at six, 12, 18 and 24 months following discharge and found that community case workers labeled most clients “successful”. This study also identifies traits that characterized successful students: Adolescents who were female, slightly younger, had higher IQ scores, better core reading and writing skills, fewer psychiatric diagnoses, and received higher parental ratings of internalized types of behaviors, were more likely to have a success rating from their case workers. In this study, these are the client characteristics that they identified as being associated with better treatment outcomes.

Lyons and Schaefer (2000) also consider individual characteristics of adolescents in residential treatment. They studied data obtained through an assessment of 15 residential treatment centers. The sample of 392 consisted of children and adolescents ages 3.3–18.5 (52% male; Lyons and Schaefer 2000). No diagnostic information was provided.

This study measured both strengths and impairments using the Child and Adolescent Strengths Assessment (CASA) and the Childhood Severity of Psychiatric Illness (CSPI; Lyons 1998; Lyons et al. 1998). Using the Danger to Others Scale of the CSPI, they assessed the dangerousness of the children and adolescents in their sample. The authors divided their sample into three groups based on violence histories: nonviolent, historically violent or currently violent (Lyons and Schaefer 2000).

This study finds significant differences between the three groups: the “currently violent” group made the most improvement in residential treatment, followed by the “historically violent”, and finally the “non-violent”. There was a significant difference in participant strengths between the “currently violent” and “nonviolent” groups, with the “currently violent” participants demonstrating fewer strengths as measured by the CASA. The “historically violent group” score fell between the “currently violent” and “non-violent” group scores and was not significantly different from either. The authors assert, “Our results suggest that residential treatment provides the greatest benefit to more dangerous cases. In part, this is due to the fact that, given their high level of mental health need, dangerous children and adolescents have the greatest room for clinical improvement” (p. 72). This study usefully identifies characteristics of adolescents appearing to benefit most from this treatment.

Another study examining treatment outcomes by client characteristics is Connor et al. (2002). This study examines 87 adolescents (mean age 13.4, no age range given) with complete admissions and discharge data who were discharged from a particular residential treatment center between 1997 and 2001.

This study looks at the abuse histories of adolescent clients. They assess outcome using the Devereux Scales of Mental Disorder (DSMD; Naglieri et al. 1994), a behavior rating scale designed to evaluate adolescent behaviors related to psychopathology. Clients’ classroom teachers completed this measure after clients had finished 1 month in the facility. Using treatment team consensus, staff filled out the Clinical Global Impressions Scale (CGI; Guy 1976), a global measure of individual outcome. While specific diagnoses were not stated, researchers indicated that all subjects had extensive mental health and psycho-educational treatment prior to admission. Researchers conducted chart reviews to assess for history of physical or sexual abuse. They coded these events as present only if there had been legal involvement as a result.

Findings indicate that some individual characteristics are associated with treatment outcome. For example, children with abuse histories had significantly more clinical psychopathology at discharge compared with children without such histories. Further, youth without abuse histories were more likely to demonstrate improvement or to have no further deterioration compared with ratings of the group with abuse histories (Connor et al. 2002). These results suggest that awareness client history may be a beneficial characteristic for a successful treatment outcome.

Curry and Craighead (1990) evaluate the attribution styles of depressed adolescents with and without a comorbid diagnosis of conduct disorder. Subjects were 63 adolescents (33 boys and 30 girls), from 12 to 18 years old, admitted to an inpatient psychiatric program. Only 50 adolescents had diagnostic assessments. Of these 50 adolescents, 27 had a diagnosis of Conduct Disorder, 10 of anxiety disorders, 18 of depressive disorders, seven of concurrent Conduct and depressive disorders, and 6 of concurrent anxiety and depressive disorders.

This study uses the Children’s Depression Inventory (Kovacs 1983) to assess self-reported depression and the Jesness Inventory (Jesness 1983) to assess self-reported conduct disorder. Researchers also used the self-report Total Anxiety scale of the Revised Children’s Manifest Anxiety Scale (Reynolds and Richmond 1985; Reynolds 1982), to assess anxiety, and the self-report Children’s Attributional Style Questionnaire (Seligman 1984) to measure clients’ beliefs about causes of events (Curry and Craighead 1990).

Results show significant correlation between the severity of self-reported adolescent depression and the failure to make internal, stable, and global attributions for positive events. However, there was no correlation between attribution style for negative events and depression. In addition, neither of the attributional style measures was related to severity of social maladjustment or anxiety (Curry and Craighead 1990). These results point to the possible influence of personal attribution characteristics on treatment outcome. This suggests that attribution is an important characteristic related to severity of depression and that treatment of residential clients should focus on addressing this trait.

Frank et al. (1997) look at the association between adolescents’ current perceptions of parental relationships and their self-critical and interpersonal concerns. In addition, they look at a possible relationship between these concerns to symptoms of depression. Sample was 295 adolescent inpatients (137 boys and 158 girls), between 11 and 17 years old, admitted to an acute care private psychiatric hospital. About two-thirds of the sample had multiple Axis I diagnoses. The most frequently occurring were Major Depression or Dysthymic Disorder (69%), Oppositional Defiant Disorder (25%), and ADHD (23%).

This study uses several self-report measures completed by adolescent participants. They used the Familial Insecurity Scale (Ainsworth and Ainsworth 1958), to meaure adolescents’ feelings of insecurity in relation to their parents and the Psychological Separation Inventory (Hoffman 1984), to measure adolescents’ psychological separation from their parents. They also used the Children’s Report of Parental Behavior Inventory (Schludermann and Schludermann 1970), to assess adolescents’ perceptions of caregivers’ parenting, the Depressive Experience Questionnaire-Adolescent Version to measure self critical and interpersonal concerns, and depending on their age, the Reynolds Adolescent/Child Depression Scale (Reynolds 1986, 1989), to assess the frequency of depressive symptoms (Frank et al. 1997).

Study results indicated associations between adolescent depression and individual self representations. “Adolescents’ difficulties with their parents are connected to adolescents’ self-representations, and adolescents’ self-representations to their experiences of depressed moods” (Frank et al. 1997, p. 213). In other words, adolescents’ perception of their depressive symptoms and their parental relationships account for a large variance in their depression. This finding appears to indicate that individual qualities impact perceived severity of mental health symptoms and in turn give direction to treatment focus.

Wise et al. (2001) evaluate 91 adolescents, ages 13–18, 61 males and 30 females, in a residential treatment substance abuse treatment program. All adolescents had a substance abuse diagnosis and 63.7% had psychiatric diagnoses as well. About 24% had a diagnosis of Major Depression and/or Dysthymic Disorder, 11% ADHD, 24% conduct disorder, 3.3% bipolar disorder, and 7.7% adjustment disorder (Wise et al. 2001). This study utilized a retrospective record review. They collected demographic and diagnostic information from client charts and recorded whether the treatment team considered each client’s program participation successful.

Results show that client characteristics may predict outcome. Female clients had a higher frequency of successful participation, and individuals with ADHD and Conduct Disorder had lower frequencies of successful participation. Similar to other studies (Colson et al. 1991; Lyons et al. 2001), these researchers find that adolescents diagnosed with disruptive behavior disorders were less likely to be seen as successful by the treatment team (Wise et al. 2001).

Studies such as these, focusing on individual characteristics, reveal the large arrays of variables affecting treatment outcome in residential and inpatient treatment. Research correlates abuse histories (Connor et al. 2002), attribution biases (Curry and Craighead 1990), relational concerns and personal perceptions (Frank et al. 1997) with outcome success. However, these studies evaluate success in different ways. Connor et al. (2002) based success on clinical psychopathology at discharge, while Curry and Craighead (1990) found that successful treatment includes a restructuring of cognitive attributions. Further, Frank et al.’s findings suggest that adolescents’ self-representations and perceptions of family relationships decrease adolescent depressive symptoms and make treatment success more likely.

Methodological Issues in Measuring Treatment Outcome

One of the greatest concerns to researchers in this field is the methodological difficulties that threaten internal, external and construct validity (Swales and Kiehn 1995). Much of the research is plagued with idiosyncratic measures, contain no comparison groups, are retrospective, and are not systematic in providing information on reliability, validity, diagnoses, symptoms, or demographics (Blanz and Schmidt 2000).

Conducting research by creating a control group is virtually impossible in this environment. Ethical guidelines prohibit withholding treatment that is known to be beneficial from individuals for the sake of research (ten Have 2005). Families often make the choice to use an inpatient or residential placement when the current living situation is unsafe for the adolescent. In such cases, random allocation to treatment or control groups presents serious ethical concerns (Swales and Kiehn 1995). Further, it is not financially feasible nor is it practical to place adolescents in residential or inpatient settings who are not in need of therapeutic intervention.

This deficit in experimental designs prevents researchers from generalizing their findings to other treatment programs or settings. For example, Leichtman et al. (2001) utilize a single-group follow-up design to examine the short-term residential stays of 123 adolescents, ages 11–18 years old (53% male). About 31% had personality disorder diagnoses, 26% affective disorders, 22% disruptive behavior disorders, and 10% psychotic disorders.

Measures for this study consisted of three interviews with adolescents or parents to collect information on adolescent symptoms and functioning. At each interview, researchers asked parents and clients to report on the client’s behavior for the previous 3-months. The first interview was at admission, the second 3 months after discharge and the third 12 months after discharge. In addition, during each interview, parents orally completed the Child Behavior Checklist (Achenbach 1991a) and adolescents orally completed the Youth Self-Report (Achenbach 1991b) to assess behavioral and emotional problems. Further, trained interviewers assessed functioning via the Child and Adolescent Functional Assessment Scale (Hodges 1996; Hodges and Wong 1996) and the Children’s Global Assessment Scale (Shaffer et al. 1983). Researchers aimed to examine symptom changes during treatment and to see whether subjects maintained changes following discharge (Leichtman et al. 2001).

This study concludes that intensive short-term residential treatment can effectively treat adolescents with severe psychiatric problems. They find statistical significant and clinically substantial improvement from admission to discharge. In addition, this study finds that the adolescents sustained these improvements for the year following discharge (Leichtman et al. 2001). However, researchers have no comparison group in this study, making these findings less robust.

One study that seeks to mitigate this dilemma, Weis et al. (2005), uses three comparison groups: graduates, drop-outs, and wait list adolescents at a military style residential treatment center. Group assignment was not random. The wait list was due to class size limitations. Their study includes a total of 242 adolescents, ages 16–18 years old, 197 boys and 45 girls, referred to a boot-camp style facility in Wisconsin. For funding and logistical reasons, this program placed 47 of the participants on a wait list. Of the original 193 participants who received residential services, 131 completed the program and 62 dropped out. The final analysis compares those who completed treatment with those who did not, both dropouts and wait list.

This study uses the parent-completed Behavior Assessment System for Children (Reynolds and Kamphaus 1998) to assess the adolescents’ behavioral functioning. In addition, researchers measured outcome using telephone surveys of the participants’ parents and teachers. Surveys took place two to 4 weeks prior to treatment. Researchers also surveyed parents at 6 months after discharge using the same instrument plus additional questions regarding adolescents’ social functioning (Weis et al. 2005).

Weis et al. 2005 conclude that adolescents who completed the program displayed significant reductions in symptoms and increases in adaptive skills from pretreatment to 6 month post-discharge. The authors did not find these changes in the wait list group. Interestingly, even those adolescents who participated in the program but did not graduate demonstrated some benefits relative to the wait list controls. Drop-out participants displayed increases in adaptive functioning, specifically increases in social and leadership skills, compared to wait list controls.

While this design provides more control than single group pre-post design, its results still have limitations (Weis et al. 2005). Wait lists are often the result of program selection criteria and lack the randomization required to establish equivalency between the control and treatment groups. In Weis et al. (2005), the program based wait list assignment on the date of the family’s program orientation and the adolescent’s gender. They made assignment in this manner to ensure the fulfillment of treatment slots for each gender (Weis et al. 2005). Dropout controls have similar issues in that they are self selected, and may indicate that a given treatment is already not working for an individual. While the non-random nature of wait-list controls is problematic, the Weis et al. (2005) study adds significantly to the residential outcome literature in its use of any comparion group. Notably, wait lists are rare in most adolescent treatment programs, as parents eager to get treatment for a child in crisis typically find treatment elsewhere when one program is full.

Cultural Issues and Confounds

The residential and inpatient outcome literature largely ignores variables of race and ethnicity in considering outcome. Some do not report ethnic demographics at all (Colson et al. 1991; Lyons et al. 2001) while a few others document the racial or ethnic make-up of their study participants (Connor et al. 2002; Larzelere et al. 2001). Most studies clump ethnicities together, reporting on Caucasian and “Other” (Brinkmeyer et al. 2004; Hooper et al. 2000; Leichtman et al. 2001; Mann-Feder 1996; Wise et al. 2001), eliminating ethnicity as a distinct and measurable variable. Vargas (1991) brings the issue of race and ethnicity in treatment to the fore. He asserts that inpatient and residential settings traditionally ignore racial and ethnic factors in their clients and have failed to be intentional in creating culturally-sensitive treatment environments. Further, Vargas asserts that psychiatric hospitalization is a solution created by a culture to deal with severe problems of living:

Clinicians who use psychiatric hospitalization for some of their patients usually are guided by a medical model that postulates the existence of discrete disease entities that can be “accurately” diagnosed with “proper” training.…This entire premise makes sense only if we believe the epistemological assumptions of the medical model, which derive from our predominant and more pervasive Western epistemologies (p. 290).

Vargas explains that the Western view of illness views physical illness as distinct from mental illness, and thus there are separate hospitals for each type. He asserts, “In a culture like that of the Navajo or the Pueblos, the dualism may seem ludicrous” (p. 290). His statements call attention not only to this cultural premise of psychiatric hospitalization, but also to culturally different thresholds in defining psychopathology. Those assessing psychopathology should be sensitive to the treatment of ethnic minority patients within a dominant culture to ensure that the belief systems of minority patients are evaluated as crucial to treatment.

The failure of the outcome literature to systematically define its samples by race, ethnicity or sexual orientation is significant. If researchers do not identify the clients for which such treatments are most effective, residential and inpatient centers will fail to have the tools they need to refine programming for maximal effectiveness. Vargas calls attention to the cultural biases inherent in residential and inpatient treatment programs. Such statements point to important directions for future research in the field, such as including diverse identity factors as variables in outcome and considering the impact of culturally sensitive programming on diverse adolescents served.

Discussion

Critics challenge the necessity and effectiveness of inpatient and residential treatment programs. Over the years, research has emerged challenging the legitimacy of these critiques. Currently, the outcome literature of adolescent residential and inpatient treatment (Table 1) indicates that these therapeutic settings are successful interventions for many clients. This is demonstrated through research measuring behavioral changes (Lyons et al. 2001) and increases in adaptive social and familial functioning (Hooper et al. 2000; Larzelere et al. 2001). In addition, certain factors such as parental engagement (Brinkmeyer et al. 2004) and individual client characteristics (Blatt et al. 1997; Curry and Craighead 1990; Lyons and Schaefer 2000) positively influence treatment outcome. Further, some research has helped to identify the importance of taking into account factors such as client history in cases where abuse might be present (Connor et al. 2002) or attribution styles when working with depressed adolescents (Curry and Craighead 1990).

Table 1 Reviewed studies of adolescent residential and inpatient treatment
Full size table

However, several significant deficits exist within the literature. One deficit is the lack of literature assessing specific programmatic elements. While adolescent residential and inpatient programs typically consist of standard program elements such as individual, group, family and milieu treatments, few studies specifically evaluate these or other programmatic elements (Zimmerman 1990). This leaves us wondering which of these programmatic elements account for success in treatment. Does individual treatment make the difference in outcome? Does group treatment contribute to outcome? Given the significant expense of multi-faceted residential and inpatient treatment, it behooves us to know which programmatic elements contribute substantively to outcome.

Another deficit in this body of literature is the lack of consensus on the definition of residential treatment (Butler and McPherson 2007). Various types of residential programs currently all fall under the same defining label. This impedes researchers’ abilities conduct systematic evaluations on the efficacy of this approach (Butler and McPherson 2007).

Lyman and Campbell (1996) further assert, “A significant weakness in much of the research literature on the effectiveness of residential and inpatient treatment is its failure to adequately specify or independently verify components of treatment” (p. 72). Many studies fail to describe residential treatment programs in sufficient detail, making it difficult to replicate treatment approaches. In many cases, program approaches are simply labeled as a “therapeutic community”, “cognitive behavioral”, etc. The lack of more precise descriptions makes it difficult to make causal attributions due to the large number of confounds that could be present.

Further, the few studies which clearly identity programmatic elements fail to define the extent to which elements are actually delivered to clients.

Another deficit in the research is the lack of operationalized definition of treatment success. Without consistent constructs of “treatment success”, it is difficult to replicate research findings. Creating systems or consistent measures of therapeutic outcome following residential or inpatient placement helps control for the possibility of attributing treatment success to irrelevant treatment variables. In addition, it would allow treatment programs to base programmatic change on research rather than continuous guess work (Lyman and Campbell 1996).

Finally, there is great need for culturally-sensitive outcome research on child and adolescent hospital treatment. The current research deficits necessitate a re-examination of the norms of evaluative research. Researchers should consider establishing standards for measuring client improvement, examining the cultural context of client behaviors, looking at the cultural norms of clients and the hospital setting, reviewing the construct validity of instruments used with minority populations (Vargas 1991), and including diverse identity factors as variables in outcome research.

This review of the outcome literature on adolescent residential treatment provides some directions for future research. Future research should utilize thus-far unutilized research designs in order to fill holes in the adolescent residential and outcome literature. Similar to Curry (1991), we assert the need for research that provides across-program and between-treatment comparisons, as well as research that utilizes quasi-experimental designs such as time-series or multiple baseline designs. Such studies would likely yield useful information regarding the value of specific interventions with this population. Since there are multiple ways of measuring outcome, researchers should utilize constructs which incorporate various theoretical perspectives and can gauge symptom reduction as well as social and relational functioning. A review of current residential and inpatient literature reveals that we know such treatment is effective for some adolescents in some kinds of treatment, but leaves much yet unclear.