Introduction

An estimated 20 % of US youth are affected by a mental health disorder severe enough to impair daily functioning. About 40 % of these youth meet criteria for at least one additional disorder (Merikangas et al. 2010). According to a study through the National Institute of Mental Health, only one-third of youth with a mental health disorder received treatment, and the majority of the children who actually received services had fewer than six visits with a provider over their lifetime (Merikangas et al. 2011). Untreated mental health disorders have financial and social impacts as they place children at a higher risk of school dropout, suicide, juvenile delinquency, and greater healthcare cost (National Alliance on Mental Illness 2014).

Outdoor Behavioral Healthcare (OBH) is a relatively new treatment modality, and has arisen to fill a gap for adolescents who have exhausted other forms of traditional treatment (Behrens et al. 2010; Gass et al. 2012; Tucker et al. 2014). OBH uses therapeutic methods typical in residential treatment settings, with the major difference being that the wilderness is both the ‘container’ for treatment and utilized as a mechanism of change. OBH is often defined as the prescriptive use of wilderness experiences provided by mental health professionals to meet the therapeutic needs of clients (Norton et al. 2014; Gass et al. 2012; Tucker et al. 2014). Critical elements of OBH include: extensive backcountry expeditions, continuous group living, primitive and wilderness living skills, and regular group and individual therapy sessions with a masters or doctoral level therapist (AEE 2014). This approach safely removes the client from unhealthy systems, peer groups, or destructive behaviors, and places them in a new and generally unfamiliar environment. Living in the wilderness and learning expedition living and survival skills builds self-efficacy and promotes personal responsibility. The group relies on the collective unit to meet emotional and physical needs, while also confronting issues, identifying denial, and supporting emotional growth. Clients also attend weekly individual therapy sessions, which may include family therapy. The use of metaphor, and a strong ethic of care and support throughout the therapeutic experience are other defining characteristics of OBH treatment (Gass et al. 2012; Tucker et al. 2014).

Over the last 20 years, OBH has improved in clinical sophistication and begun to build a base of evidence. A number of OBH outcome studies demonstrate that clients enter OBH programs with high clinical dysfunction, make dramatic improvements in emotional and behavioral functioning while in OBH treatment, and maintain improvements up to 6 and 12 months post-OBH treatment (Behrens et al. 2010; Bettmann et al. 2013; Hoag et al. 2013; Lewis 2013; Magle-Haberek et al. 2012; Norton et al. 2014; Russell 2003, 2005; Tucker et al. 2014).

The first major OBH study looked at parent and youth assessments from 858 participants at seven different OBH programs. This study showed that clinically and statistically significant improvements were made during OBH treatment, and that clients maintained these positive results 1 year after discharge (Russell 2003). Two years later, the researcher interviewed a random selection of participants and found that 83 % reported to be doing better, 58 % said they were doing well or very well, and 81 % rated OBH treatment as effective (Russell 2005).

Recent OBH literature continues to support that OBH is effective for a wide range of presenting problems and complex issues (Behrens et al. 2010; Lewis 2013; Hoag et al. 2013; Norton et al. 2014; Russell 2003, 2005; Tucker et al. 2014). In 2013 Lewis conducted a study with 190 participants from three OBH programs using growth curve modeling. The author found that improvements in substance and behavioral issues continued, rather than faded, across the 3- and 12-months post-treatment follow-ups. These improvements remained true regardless of whether a youth attended additional treatment (Lewis 2013). Bettmann et al. (2013) conducted a study with 189 adolescents from one wilderness therapy program. They found that OBH reduced mental health symptomology for a wide range of problem behaviors, and that clients reported sustained positive outcomes and increased coping skills around abstinence at the 6-months follow-up (Bettmann et al. 2013). The National Association of Therapeutic Schools and Programs (NATSAP) created a Research and Evaluation Network with the goal of evaluating the effectiveness of member programs. Tucker et al. examined this data to see how pre-treatment youth characteristics impacted outcomes at discharge for 896 adolescents in OBH programs. They found that clients made large improvement from intake to discharge, and that gender, rather than presenting issue, was the only significant predictor of change (Tucker et al. 2014).

While OBH has made great gains in building an evidence base, several important limitations exist: no control or comparison groups, participant attrition at post-discharge, few studies using sophisticated methods of analysis, limited programs represented in published research, and inconsistent data collection resulting in the loss of demographic and diagnostic data. Additionally the vast majority of OBH studies have used only adolescent self-report measures (Becker 2010; Bettmann et al. 2013; Lewis 2013; Norton et al. 2014; Russell and Sibthorn 2004; Tucker et al. 2014; Tucker et al. 2011). While the field has numerous studies documenting impressive changes among adolescent self-reports during treatment, only a few studies have been able to speak to what influences this change, if the change lasts after treatment, and if parents also report improvements.

This study sought to extend the literature by identifying treatment outcomes and predictors of change during OBH treatment and up to 18 months post-discharge according to the parent perspective. This study is unique in that it included a parent-report measure, data up to 18-months post-discharge, robust demographic and diagnostic data, and a rigorous method of analysis (multilevel modeling).

Our first two hypotheses related to change during OBH treatment and the second two related to change after OBH treatment. The first hypothesis was that parent reports of change would confirm existing findings among adolescent self-reports that clients make significant improvements in behavioral and emotional functioning during treatment. The second hypothesis was exploratory and sought to identify predictors of in-treatment change. The literature has consistently found that gender predicts adolescent self-reports of change during treatment, and that presenting issues have not been significant predictors. However, study designs have limited the ability to speak to this question and parent-reports have not been recently examined in the literature.

Our last two hypotheses relate to outcomes after OBH treatment. The third hypothesis was that parent-reports of wellbeing would support the existing literature of adolescent self-reports affirming that on average clients maintain healthy functioning post-discharge. The fourth hypothesis was exploratory as the literature has yet to explore the question of diagnostic and demographic predictors of change and outcome post-discharge on parent reports.

Methods

Participants

This study used a quasi-experimental design with no control group, and included 659 parents of adolescents who enrolled in one of four wilderness therapy programs between June 2011 and June 2012. The wilderness programs were located in southern Utah, northern Utah, Oregon, and north Georgia, and were connected by ownership and management. Of the 792 adolescent clients who entered the four programs, 88 refused to participate, and 45 were excluded due to leaving the program before completing 5 weeks (83 % participation rate). Thirty-two percent of adolescents in the sample were female, 18 % were adopted, and 65 % had parents who lived together. The average age was 16.3 years and the average length of stay in the program was 10.4 weeks.

This analysis focused on a smaller subsample of 338 clients whose parents had completed assessments at intake and discharge. Due to the extensive resources required to successfully follow-up post-treatment, 200 participants were randomly selected for the 18-months follow up. Of this group, 99 youth clients had parents who completed post-treatment assessments at discharge and at one of 6 or 18-months post-treatment. Therefore, the MLM on post-treatment change is a subsample of the original study population. The demographics of this subsample were similar to the main sample with an average age of 16.2 years, the average length of stay 10.4 weeks, 16 % were adopted and 31 % were female.

Table 1 displays data on client presenting problems for the main and post-treatment samples. Data on ethnicity or socio-economic status was not collected in this study; however, participants in OBH programs tend to be Caucasian and from higher income families (Russell et al. 2008). A past review of clientele at the program located in southern Utah indicated that 85 % of clients were identified as white or Caucasian (Hoag et al. 2011).

Table 1 Presenting problems for main and post-discharge samples
Full size table

Treatment Model

The OBH programs in this study used an open-group, traditional wilderness therapy model. Characteristics of this model include that the group engages in a nomadic or expedition hiking plan; as well as an experiential approach utilizing skills such as bow drill fire making, cooking over an open flame, and building shelters in the forest or high desert. Treatment elements include aspects of Cognitive Behavioral Therapy, Choice Therapy, Family Systems, mindfulness techniques, and a focus on diet and physical exercise. The ‘wilderness’ sites for these programs were undeveloped areas of land typically administered by the Bureau of Land Management (BLM) or the U.S. Forest Service (USFS). Clients remained in the wilderness for the entirety of their stay with the wilderness being used as a critical therapeutic tool. Yalom and Leszcz (2005) group factors provide a framework for understanding the process of the model. Group cohesiveness occurs as the participants work through challenges together, and relationships in the wilderness parallel the process they have with their parents, siblings, and peers. Much of the program is designed to assist with accessing cathartic experiences, through bow drill fire making, hiking, or other camping skills. General treatment goals include crisis management, individualized treatment process, assessment in a natural substance free environment, developing coping skills, improving self-esteem through task accomplishment, focused family systems work, and social skills development.

Procedures

This study utilized data from the OBH programs’ internal efforts to monitor client progress and assess treatment outcomes. The program obtained informed consent from each participating parent, and explained that there was no obligation to participate. Consenting parents signed a release stating that their child could be a part of this internal project and also giving permission for the program to use theirs and their child’s data for professional or research purposes, given that it was de-identified. Three of authors were employees of the program, and the fourth author only accessed de-identified, aggregate-level data.

Outcome Tools (an online data storage program) was used to collect and store data. Demographic information on adoption, parent marital status, and age was pulled from the client’s application to the program. Due to it being the most accurate diagnostic source, diagnostic data was collected from discharge summaries completed by the therapist upon discharge. The first four diagnoses listed on the discharge summary for each client were recorded. Category titles follow DSM diagnostic categories, including mood disorders, substance-use disorders, anxiety disorders, behavioral disorders, attachment disorders, pervasive developmental disorders, and learning disorders.

All parent data were collected electronically, through emailed links, allowing participants to submit questionnaires online. Data were collected at intake, discharge, 6-months post-discharge, and 18-months post-discharge (random selection of 200 at the 18-months follow-up). Participants were not offered an incentive for participation, and the post-discharge response rates were 69 % at 6 months and 67 % at 18 months. The response rate for matched post-treatment parent data (discharge and one of 6 or 18-months follow up) was 49.5 %.

Measures

The Youth Outcome Questionnaire™ 2.01 (Y-OQ) was used to assess treatment progress and outcome. The Y-OQ is completed by the parent or guardian and is designed to measure the psychological and behavioral symptoms as well as social functioning of adolescents. It includes 64 items that make up six subscales: Intrapersonal Distress, Somatic Symptoms, Interpersonal Relations, Social Problems, Behavioral Dysfunction, and Critical Items, (Burlingame et al. 2005; Wells et al. 1996). The measure is easily administered and has high internal consistency and test-retest reliability. Using the Cronbach’s alpha, Burlingame et al. (2001) estimated the Y-OQ total score internal consistency to be .94 among non-clinical and clinical samples. Test-retest reliability coefficients were .83, indicating high temporal stability (Burlingame et al. 2001).

The Y-OQ has a reliable change index (RCI) and a clinical cutoff score. As statistical significance does not always equate to clinical significance, the reliable change index (Jacobson and Truax 1991) identifies whether the magnitude of change is clinically significant. The Y-OQ defines scores below 46 to be in the community or normal range of functioning, and a change of 13 points or greater to be evidence of clinically significant change. When cutoff scores are reached and the reliable change index of 13 points is met or exceeded a client may be labeled “recovered” (Burlingame et al. 1996, 2005).

Analysis

For the first hypothesis, a t test was conducted to examine change from intake to discharge. For the second hypothesis, an OLS multiple regression was employed to investigate factors associated with parent change scores from intake to discharge. Hypotheses three and four were addressed using an MLM to estimate prototypical change trajectories for Y-OQ scores from discharge through 18 months post-treatment. This approach is ideal for examining longitudinal data, and also allows for variations in data collection across the sample. It is acceptable for some of the sample to be missing data at some time points, and for subjects to have varying data collection schedules (due to varying discharge dates) (Singer and Willet 2003). Parent data was included in the MLM if parents completed intake and discharge questionnaires, and had at least discharge and 18 months follow-up data. Matched data across all four parent data collection points was available for 99 clients. Independent sample t-tests were conducted to compare all demographic, presenting problem, and Y-OQ score data between the main sample and the post-discharge follow-up sub-sample. Bonferonni corrections were not used to increase the chance of finding a significant difference between groups. No significant differences were found between these groups on any variable. As a result, it was reasonable to assume that the post-discharge subsample was representative of the larger sample.

Results

Descriptive Statistics

Descriptive statistics of the in-treatment sample and post-discharge sample are displayed in Table 2. On average, parents reported that clients entered treatment with significant levels of emotional and behavioral dysfunction (M = 97.55, SE = 1.22, SD = 27.26), made clinically significant improvements through treatment, and discharged in the normal range of functioning (M = 32.90, SE = 1.57, SD = 31.32). Parent post-discharge data also showed that on average, clients showed slight increases in symptom severity following treatment, but these changes were not clinically significant, and clients were still in the normal range of functioning at six (M = 32.54, SE = 2.92, SD = 29.07) and 18 months post-discharge (M = 38.05, SE = 3.48, SD = 35.91; Table 3).

Table 2 Y-OQ 2.01 descriptive statistics
Full size table

Hypotheses 1 and 2: Change During OBH Treatment

A t test showed that there was significant change in client functioning through treatment. On average, parents reported improvements of 65 points from intake to discharge (t = 35.0, p = 0.000, df = 337), far surpassing the reliable change index. An OLS regression analysis was conducted to better understand what factors affect parent reported change from intake to discharge (referred to as “parent change score”). The independent variables included diagnosis, client gender, client age at intake, length of stay, adoption status, and whether the client’s parents were living together or separately. While the regression model (Table 3) significantly predicted change in parent Y-OQ scores from intake to discharge (F = 13.91, p = 0.001, R 2 = 0.32), only two of the eleven included independent variables were found to be significant predictors of parent change scores individually. On average, clients whose parents reported more severe problems at intake showed greater improvements in functioning at discharge regardless of mood, substance use, anxiety, behavioral, and attachment disorder status, length of stay, age at intake, parent’s marital status, and adoption status (B = 0.68, SE = 0.06, t = 11.7, p < 0.001). Controlling for all other variables in the model, clients presenting with attachment disorders tended to experience lower treatment gains (B = −29.31, SE = 8.16, t = −3.59, p < 0.001).

Table 3 OLS regression predictors of parent-reported change in problem severity from intake to discharge (n = 338)
Full size table

Hypotheses 3 and 4: Change Post OBH Treatment

A series of MLMs for change in Y-OQ scores were fitted with an increasing number of predictor variables in order to establish the model of best fit. Empirical growth trajectories (OLS regression) were created for five randomly selected cases. This initial inquiry showed variation in both Y-OQ scores at discharge and rate of change in Y-OQ score from discharge through 18 months post-treatment.

An unconditional growth model fitted to the data showed that while there was significant variance in initial status (discharge parent Y-OQ score), there was not significant variation in rate of change (ROC) in Y-OQ scores from discharge through 18 months post-discharge. This does not imply that there is no variation in ROC in post-discharge scores, rather that this variation was symmetrical in the sample, and effectively balanced itself out. As a result, ROC was modeled as a fixed effect (i.e. not effected by any independent variables). A second unconditional growth model was created and served as the baseline for comparison of subsequent models.

A final model was created to evaluate the effects of the following variables on Y-OQ change trajectories from discharge to 18 months post-treatment: diagnosis, adoption, aftercare, parent marital status, client age at intake, gender, and parent change scores from intake to discharge (Table 4). Controlling for all other variables in the model, parent change scores (γ 01 = −0.47, SE = 0.08, p < 0.001) and adoption status (γ 010 = 17.00, SE = 7.32, p = 0.023) were the only significant predictors of post-discharge Y-OQ scores. On average, youth whose parents reported greater change scores from intake to discharge were estimated to have lower Y-OQ scores at post-discharge follow-up, while youth who were adopted were estimated to have higher Y-OQ scores. ROC in Y-OQ scores post-treatment was modeled as a fixed effect, and was thus consistent for all clients. On average, all clients were estimated to show an increase of about two points in Y-OQ score every 5 months post-treatment (γ1 0 = 0.42, SE = 0.18, p = 0.020). While there was not sufficient data to model variation in ROC in Y-OQ scores across post-discharge data points, the final model (\( \sigma_{0}^{2} = 361.63 \), SE = 86.44, p < 0.001) accounted for 34.53 % of the variance in estimated initial status (discharge Y-OQ score) present in the unconditional growth model (\( \sigma_{0}^{2} = 445.76 \), SE = 123.00, p < 0.001).

Table 4 Estimated change trajectories of parent-reported scores from discharge to 18 months post-treatment (n = 99)
Full size table

The final model illustrated in Fig. 1 predicted that while there was some regression in treatment effects 18 months post-treatment, on average, it was less than the reliable change index of 13 points. This indicated that the average decrease in functioning was not significant, regardless of diagnosis, age, time in treatment, adoption status, or parent’s living conditions.

Fig. 1
figure 1

Prototypical post-discharge change trajectories for male clients of average age presenting with mood disorder by adoption status at low, mean, and high levels of parent reported change from intake to discharge (N = 99)

Full size image

Discussion

Parent-reports in this study paralleled adolescent self-reports in the literature demonstrating positive change during treatment and maintenance of these improvements post-treatment. This study makes an important contribution to the OBH literature by adding parental reports, longitudinal data, and rigorous methodology. According to parents on the Y-OQ, the OBH participants entered treatment with clinically significant levels of emotional and behavioral dysfunction, made clinically significant change, and discharged within the “normal” range of functioning. Post-discharge scores indicated that on average, while there was some increase in symptom severity following treatment, it was not clinically significant, and clients remained in the normal range of functioning at 6 and 18 months post-discharge.

Change During OBH Treatment

The regression model suggested that intake scores and attachment related disorders predicted parent perceptions of change during OBH treatment. Clients with greater dysfunction at intake tended to experience greater positive change during OBH treatment. This finding is in-line with previous work by Magle-Haberek et al. (2012) who found that clients entering with greater dysfunction self-reported larger change scores. The regression model also estimated that while the average client with attachment-related disorders showed clinically significant improvements in functioning through OBH treatment, the gains were not as substantial as clients with other diagnoses. Diagnosis or presenting issue has not been found to influence change scores previously in the literature. While this study is the first to find a significant diagnostic predictor, the fact that the four major diagnostic categories (mood, anxiety, behavior, and substances) were not significant predictors of treatment outcomes, continues to suggest that OBH treatment is helpful for adolescent clients with a wide variety of presenting problems.

While the regression model predicting parent-reported change scores was significant, it only accounted for 32 % of the variation in parent change scores. This means that 68 % of the variance in parent-reported change through the OBH treatment is due to other factors. To better understand the variables that influence treatment outcomes, future research should explore personal and program-level characteristics beyond those included in this study.

Previous OBH research using adolescent self-report data has shown differences in outcomes by gender. Using the Y-OQ, Russell (2003) and the NATSAP Research and Evaluation Network (Tucker et al. 2014) found that females scored higher at admission and made greater overall change during treatment in OBH. This trend was reflected in a study among young adults in wilderness therapy (Hoag et al. 2013) and in a study on adventure therapy in community-based treatment (Tucker et al. 2013). However, in the current study using parent-reported data, females had similar change scores as males and gender was not a significant predictor of change during or after treatment. Future research looking at differences between males and females on subscales, and exactly where and how males and females are changing would shed light on this issue. This study highlights Tucker et al. (2013) conclusion that “future research is needed to explore in more depth why or in fact if this modality truly impacts youth differently based on gender” (p. 174).

Change After OBH Treatment

While these findings are congruent with previous work exploring post-treatment functioning in OBH (Bettmann et al. 2013; Lewis 2013; Russell 2003), they also add several new insights. This study corroborates findings from several studies that OBH clients maintain progress post-treatment. While the OBH field has limited data post-discharge, what is available is promising and shows maintenance of treatment effects as reported by adolescents (Bettmann et al. 2013; Lewis 2013; Norton et al. 2014; Russell 2003). Using a parent-report measure, this study affirms such positive findings from adolescent self-reports, and goes 6 months further than any other quantitative follow-up in the literature. Twelve months was the longest, quantitative follow-up in the published research (Russell 2003); though many of the follow-ups in the literature are 6-months post-discharge. As many OBH clients go onto other levels of therapeutic care that often last 12 months, this OBH program specifically chose an 18-months follow-up in order to reach clients when they would likely be out of therapeutic care settings. This study suggests that clients are remaining healthy (according to the Y-OQ) even when they are not in a therapeutic setting and for longer than what has previously been shown in the literature.

Additionally, this study begins the process of understanding what predicts long-term outcomes. The MLM showed that parent change scores during OBH treatment and adoption status predicted functioning post-discharge. Clients whose parents believed that their child made the largest changes during treatment were more likely to stay within the healthy range of functioning (<46 on the Y-OQ) up to 18 months post-treatment. While adopted clients did not experience any greater rebound in treatment gains post-discharge, they were more likely than non-adopted clients to break back into the range of clinically significant emotional and behavioral dysfunction.

Though this study was unable to model variation in ROC post-treatment, it illustrated that the participant regression experienced upon leaving OBH treatment is minimal and does not surpass the requirements for the Y-OQ’s reliable change index (13 points, Burlingame et al. 2005). While some gains in functioning made during treatment may be lost, on average, the regression is clinically insignificant. It was of interest that the variance in ROC following treatment was found to be insignificant in this study. While it is possible that there are no differences in clients’ ability to maintain treatment gains regardless of aftercare, age, gender, or diagnosis following OBH treatment, preliminary visual analysis of individual cases showed this was not the case. Rather, it is likely that the sample size in this study was insufficient to accurately model these differences. Future research should continue to explore the application of change trajectory analysis with larger sample sizes to better understand the factors that may affect maintenance of treatment effects.

Limitations

There are several important limitations to this study. Namely, this was a quasi-experimental design with no control group, which inhibits generalization of results and the ability to establish efficacy of an intervention. Additionally, the four wilderness programs represented in this study were connected by management and ownership, and were private-pay, traditional wilderness therapy models of OBH treatment. Therefore, the findings are limited and do not represent all OBH treatment, or even all traditional wilderness therapy models. While this analysis can provide considerable insight into parent views on client progress in OBH treatment and the maintenance of treatment effects post-discharge, the post-discharge sample was not large enough to model variation in ROC following treatment. It also only represents parents who chose to complete the questionnaire post-discharge. The post-discharge response rates were close to 70 %, leaving over 30 % of parents missing from this group. As a result, it is possible that these results are inflated. It is also possible that the highly dysfunctional scores at intake on YOQ are influenced by a parent’s desire to justify enrolling their child in an expensive treatment program or because they are dealing with a traumatic or overwhelming moment in their life as a parent as they admit their child to residential treatment. Furthermore, there is considerable within-person variation in Y-OQ scores across time, suggesting the need to include time-varying predictors in analyses. Future research should include additional program level and time-varying predictors to better account for these concerns.

Conclusions

This study makes an important contribution to the OBH literature by adding parental reports, longitudinal data, and rigorous methodology. It yielded positive results that affirm the dramatic changes documented in adolescent self-reports and went farther than any other quantitative follow-up in the literature showing maintenance of treatment effects. This study also contributes to understanding diagnostic predictors of outcome in OBH during treatment and post-treatment. According to parental reports on the Y-OQ, attachment and adoption were identified as predictive of parent-reported change. While gender has been the major predictor variable in adolescent self reports, it was not significant among these parent-reports. Future research should focus on securing larger post-discharge samples as well as program level variables to better model and further understand treatment outcomes.