Systems of care were developed in response to the need for more appropriate and accessible preventive and treatment services for children with severe emotional and behavioral difficulties and their families. In 1992, the United States Congress established the Comprehensive Community Mental Health Services (CMHS) for Children and Their Families Program, which has provided funding to 126 communities over the past 14 years for the development of local systems of care.1 A system of care is a coordinated network of community-based services and supports that is created to meet the challenges of children and youth with serious emotional disturbance and their families. Central to the philosophy of systems of care are community-based alternatives to out-of-home placements, family involvement, cultural sensitivity, and interagency collaboration.2 As a result, system of care communities offer an array of wraparound services individualized to the families’ needs. These services vary by site, but may include assessment and evaluation, case management, outpatient therapy, inpatient services, intensive home-based care, respite care, therapeutic foster care, vocational training, and juvenile justice services.

More than 70,000 children and their families have received services through the CMHS Program.3 Research on these systems has shown some mixed effects. For example, one study revealed that although service access (the ratio of services received relative to those recommended upon service entry) and amount of services received increased in a system of care, children who did not receive any services improved at the same rate as children who received services.4 In contrast, a study demonstrated that the degree to which a child and family reports that services were consistent with a system of care philosophy was associated with fewer internalizing and externalizing symptoms in the child and greater family satisfaction 1 year after receiving services.5 Similarly, Foster et al.1 compared two CMHS-funded system of care sites to two matched communities not implementing systems of care and found that the system of care communities provided more family-focused care, supportive collaboration, individualized plans, adequate access, and less restrictive services. Finally, Tebes et al.6 examined a behavioral health system of care over an 8-year period and found that service access increased, while barriers to service access decreased over time once the system of care was fully implemented.

Although research has revealed some positive results regarding implementation of and outcomes for children in systems of care, several risk factors associated with deterioration and/or negative outcomes in systems of care have also been identified.7–10 Because youth and family risk factors have been found to impact outcomes in systems of care and access to services is also related to youth outcomes in systems of care,1,5,6 the purpose of the present study was to investigate the relationship between youth and family risk factors and service system variables (e.g., service access, recommendations, delivery, dosage) in a school-based system of care.

Risk Factors and System of Care Outcomes

Youth and family substance use histories

Histories of substance use by youth and their families have been identified as risk factors for increased substance abuse, mental health symptoms, and negative outcomes in systems of care.8,9 Alcohol and drug abuse during adolescence remains a serious public health problem. Early onset of drug use has been consistently found to predict subsequent misuse and abuse of drugs. Moreover, poverty and childhood behavioral issues—two characteristics that typically describe youth and families being served in a system of care—have been shown to collectively increase the risk of later alcohol and drug problems.8

A study on a San Diego system of care found that 70% of enrolled 13- to 18-year-olds received some kind of mental health service. However, unmet need for services was greatest among the 37% of youth who had a substance use disorder, even after controlling for the effects of other socio-demographic and family variables known to predict service use (age, gender, race, caregiver strain, and police contact). The authors argued that this apparent lack of attention to youth with substance use disorders represents a tremendous concern.11 This concern becomes even more prominent when considering the assertion that approximately half of adolescents receiving mental health services in the general population are dually diagnosed. Among the juvenile justice system, the rates may be even higher12,13 as many youth with such comorbidity fall under the jurisdiction of juvenile justice systems, leading to high rates of unmet need for their mental health and substance use problems.14 Adolescents with comorbid psychiatric and substance use diagnoses may present more of a challenge for systems of care to serve effectively due to higher levels of functional impairment and higher costs for service.15

Youth’s history of suicide attempts

An examination of data from the national evaluation of the CMHS Program compared suicidal and non-suicidal youth (see Holden et al.16 for a description of the CMHS evaluation). Suicidal youth were defined as those whose caregivers or parents had indicated at least one suicide attempt by the child. Results revealed that youth with a history of multiple suicide attempts were more likely to be functionally impaired in several domains when compared to youth with no history or a history of just one attempt, and these differences in impairment persisted 6 months following service entry. Moreover, those youth with a history of suicide attempt(s) who did not evidence severe functional impairment at service entry were more likely to exhibit severe functional impairment at 6-month follow-up. Youth who had attempted suicide were also more likely than other youth to have experienced a number of other risk factors at service entry, including physical abuse, sexual abuse, and substance use.17 The results of this and other studies18,19 suggest that a history of suicide attempts places youth at higher risk for lower levels of functioning and may predict deterioration for youth receiving services within a system of care.

Youth racial/ethnic background

Another study utilizing data from the national evaluation of the CMHS Program examined pre-referral characteristics of children and youth entering systems of care and revealed that minority racial/ethnic background status predicted deterioration in problem behaviors and behavioral and emotional strengths 6 months after entry into services. Specifically, youth of color were four times more likely than similar white children to deteriorate while in system of care service. These differential outcomes did not appear to be due to differences in income, gender, or services received.10 The results highlight the importance of culturally competent services, a tenet of systems of care, and the potentially elevated risks for youth of color to exhibit more difficulties after entry into system of care services.

Youth’s history of out-of-home placements

Out-of-home placements also place children and youth served in systems of care at higher risk for negative outcomes. Walrath et al.10 found that almost half of the children who deteriorated over the first 6 months in the system of care had a history of out-of-home placement in the 6 months prior to service entry. Therefore, these children were twice as likely to deteriorate in the first 6 months compared with children who did not have a history of out-of-home placements, suggesting that this history may be a risk factor for negative outcomes in systems of care.

Family history of mental illness

Numerous studies have found that children of parents with a mental illness are at increased risk for interpersonal difficulties, behavioral problems, and mental health symptoms.20–23 The elevated risks for these children may be due to a variety of factors related to a caregiver’s mental illness, including repeated parental hospitalizations, impaired parenting skills, and genetic traits.20,24 In addition, family mental illness is associated with increased likelihood of single parenthood, household discord, poverty, housing difficulties, and substance use.20,25 Because these associated factors are often present in the populations served by children’s mental health systems of care, it follows that a history of family mental illness could also negatively impact these youth.

Service System Variables in a System of Care

Youth outcomes in systems of care have been found to be influenced not only by risk factors, such as those previously identified, but also by components of the service system (e.g., service access, recommendations, delivery, dosage).26 However, system of care sites differ in terms of age and needs of the children served, how they enter the system (e.g., mental health agencies, juvenile justice system, etc.), services offered, and the service setting (e.g., school, agency, or home).1 For example, services offered can include mental health (e.g., care coordination, inpatient, outpatient, and therapeutic mentoring), operational (e.g., emergency funds, family advocacy, and legal services), juvenile justice (e.g., intensive case management), or social services (e.g., family preservation or reunification, foster family support, and shelter). Regardless of the types of services offered, a primary objective in any system of care is access to appropriate services. Systems of care that have been able to target needs effectively have generally increased service access over previous levels of service participation.6

Currently, there are few system-level indicators of service system change and development.10,27 Recent research has suggested that two indicators of service access, the number of services recommended upon service entry and the number of services a youth or family receives, may be useful benchmarks of service system development for systems of care.6 For example, in an analysis of the national evaluation of the CMHS program, children who demonstrated deterioration 6 months after entry into the system of care were significantly more likely to have received no services in the prior 6 months compared to those children who had improved.10 However, service utilization did not appear to contribute to deterioration over time when other risk factors (e.g., minority racial/ethnic background, and history of substance use or out-of-home placement) were included in the model. Nevertheless, this study did not investigate number of services received, dosage of services, or broader categories of services other than those traditionally defined as mental health. Specifically, Walrath et al.10 examined service categories such as case management, outpatient, inpatient, and support, but did not include services such as juvenile justice, educational, or operational services. Moreover, they did not assess outcomes when youth did not receive services that were recommended. An examination of both services recommended and services received, as well as dosage of services received, may provide more insight regarding how systems of care are operating and how children and youth receiving services progress. In addition, assessing how pre-identified risk factors for deterioration might impact these service system variables will provide valuable information.

Purpose of the Study

In response to research revealing that both risk factors and service system variables are related to youth outcomes in systems of care,1,5–10 the purpose of this study was to address the lack of research regarding how pre-referral risk factors proximally impact service-level decisions and outcomes. Specifically, this study sought to examine if the presence of these risk factors was associated with the number of services recommended and received and the dosage of services received in a school-based system of care. Although services recommended and received may be highly correlated, a goal of this study was to determine if there were any differences for these service variables based on risk factors; therefore, both service system indicators were included in the analyses. In addition, the current study intended to identify which risk factors predicted whether or not a youth received different types of services, including mental health, operational, juvenile justice, and social services. Although the rate of receiving these types of services may be correlated, it was hypothesized that risk factors would be differentially associated with types of services received. Because service system access has been found to be related to youth outcomes, it is important to identify if youth and family risk factors are also associated with service system access. Therefore, the aim of this study was to elucidate how youth and family characteristics influence service decisions and service system components. It is hoped that an increased understanding of service system operations in systems of care will inform the design and refinement of service planning and implementation.

Method

The PARK Project

The Partnership for Kids, or PARK, Project, funded by the Substance Abuse and Mental Health Services Administration’s (SAMHSA) Center for Mental Health Services as part of the CMHS program, is an innovative approach to community-based service delivery through partnership with local schools, families, providers, and state agencies for the purpose of producing positive outcomes for children and youth with serious emotional and behavioral challenges. Unlike many systems of care that provide services through mental health agencies, the PARK Project provided services in and through the schools. The mission of the project was to build a system of care in partnership with home, school, and community so that children with behavioral and mental health challenges can achieve success.

The PARK Project focused on systems change by developing true partnerships between parents, youth, service agencies, and schools. In order to be eligible for enrollment in the PARK Project, a youth had to (1) be attending one of the targeted schools, (2) have a DSM-IV diagnosis, (3) be in need of multi-agency services, (4) be at risk for or in out-of-home placement, and (5) exhibit impairment in school, home, and/or community that has lasted longer than 1 year. All families enrolled into the PARK system of care received school-based care coordination services and an array of wraparound services individualized to the families’ needs; possible services included mental health (e.g., care coordination, inpatient, outpatient, home-based, testing, medication, and therapeutic mentoring), operational (e.g., emergency funds, family advocacy, parent training and consultation, and legal services), educational (e.g., tutoring, adult education), recreational (e.g., after school programs, specialized camps), vocational (e.g., job placement assistance), health (e.g., primary care, dental), juvenile justice (e.g., intensive case management), and social services (e.g., family preservation or reunification, foster family support, financial counseling, and shelter or a group home).

All PARK families worked with a care coordinator whose role was to facilitate child-specific team meetings for each family, during which service providers and natural supports gathered with the family to identify family strengths, areas for growth, develop service goals and a service plan, and assess family progress. Care coordinators also worked with families to identify service providers and access services, obtain entitlements, and provide support to the families (e.g., attend IEP meetings with the parent).

Participants and procedure

All families enrolled into the PARK Project were invited to participate in a longitudinal outcome study. Families who elected to participate were interviewed in their homes or a location of their choosing when they first entered services (baseline) and every 6 months for up to 3 years. Although the average length of stay in services was 7.95 months (range = 0.5–27 months), families continued to participate in the outcome study interviews after service receipt ended. The PARK evaluation team employed four to six outcome study interviewers, half of whom were parents of children served in the system of care, to conduct home-based follow-up interviews with youth aged 11 and older and their parent or caregiver. The families received a $40 gift card for participation in each interview. The Human Investigations Committee at the Yale School of Medicine provided oversight of this evaluation with regard to the protection of study participants. A total of 194 PARK families (64.9%) elected to participate in the longitudinal outcome study. The sample for the current study was restricted to children 11 and older, resulting in a total of 125 families.

Measures

Several measures were included in this study, including youth and family demographics, youth and family risk factors, and service system variables. All measures were youth or parent/caregiver report and were required data elements in the ORC MACRO evaluation of the Comprehensive Community Mental Health Services for Children and their Families Program funded by the SAMHSA Child and Family Branch.16 The services system variables were collected as part of the local evaluation by care coordination staff.

Youth and family demographic characteristics

Youth and family demographic characteristics were obtained from the Enrollment Demographic and Information Form, which was completed with information provided by the caregiver. Demographic characteristics included gender, age, race, ethnicity, and household income.

Youth and family risk factors

Caregivers provided lifetime histories of the youth and family on the Caregiver Information Questionnaire (CIQ) at intake. The caregivers responded yes or no to indicate the presence or absence of risk factors. The risk factors included parent/caregiver history of substance abuse, family history of mental illness, youth’s history of out-of-home placements, youth’s history of suicide attempts, and youth’s history of substance use, which was a composite variable from caregiver’s report on the CIQ and youth’s responses to the Substance Use Survey-Revised. Due to the overwhelming majority of youth of color in this sample (81.2%), this risk factor was not included in the analyses.

Services recommended and received

Service system variables were obtained from the Resource and Outcome Data Form (RAODF)6 which was developed to document system-level indicators and outcomes. The RAODF allows for check-off documentation of services across the seven domains (e.g., mental health, social services) listed earlier. Care coordinators completed the RAODF during child-specific team meetings at the initial case review meeting and tracked the recommended services for 3 months to determine if they were received. As a result, the information collected was summed across domains to yield the number of total services recommended and the number of total services received. In addition, dichotomous variables indicating whether or not four specific types of services (mental health, operational, juvenile justice, and social services) were received were created for this study. The juvenile justice service variable and total services recommended did not include probation or parole because these services are mandated.

Service system variables were available for 85 out of the 125 families. For this sample of 85, the number of total services recommended ranged from 0 to 17 (mean = 7.59), and the number of total services received ranged from 0 to 13 (mean = 4.44). Total services recommended and total services received were significantly, but not completely, correlated (r = 0.61). The dichotomous services variables indicated that 77.6% of the sample received mental health services, 59.7% received operational services, 9.7% received juvenile justice services, and 59.7% received social services. Descriptive information for the major service types and total services are presented in Table 1.

Table 1 Means, standard deviations, ranges and total count of the types of services included in the analyses
Full size table

Dosage of total services

Program staff from all PARK Project-funded programs documented dosage information on an ongoing basis until a child exited the system of care. Services were logged by type and length (15-min increments) and the data sent to the evaluation team on a quarterly basis. The dosage variable includes the sum total of dosage information for all programs funded by the PARK Project. Therefore, the dosage variable represents the full dose of system of care-funded services received by a child and his/her family. Service dosages ranged from 0 to 402.67 with a mean of 92.58 (SD = 8.93, N = 81). Due to a substantial positive skew of the distribution, a square root transformation was computed for this variable prior to analysis.

Sample

The sample for the study was restricted to the 85 youth who had service variable data available. A preliminary ANOVA revealed that the 40 youth without service data did not significantly differ from the 85 with service data on any of the risk factors or demographic variables, with the exception of age in which the mean age for youth with service data was 13.85 and the mean for youth without the data was 15.02. The majority of the sample was male (62%) and Hispanic (65%). Caregivers identified the youth as 35% Black or African American, 31% other (most of whom were Hispanic), 8% White, and 1% Native American. Thirty-three percent of the caregivers reported a history of family substance use; 68% reported a family history of mental illness; 11% reported that the target youth had attempted suicide in the past; 6% reported that the youth had a history of out-of-home placement; and 22% reported that the youth had a history of substance use. Characteristics of the sample in terms of demographics and risk factors are displayed in Table 2.

Table 2 Demographic characteristics and risk factors for the sample
Full size table

Data analysis

In order to examine the impact of child and family risk factors on the service variables, two Poisson regression analyses were employed with five risk factors (family substance use history, family mental illness history, youth history of suicide attempts, youth history of out-of-home placements, and youth substance use history) as predictors. The outcome for the first regression analyses was a count of the total number of services recommended; the outcome for the second regression analyses was a count of the total number of services received. To examine the impact of the risk factors on dosage of services received, a multiple regression was conducted with the five risk factors as predictors and the square root of dosage as the outcome variable. All three regression analyses were calculated using the MPlus statistical package28 which uses full information maximum likelihood to manage missing data.28

To identify which risk factors predicted whether or not a youth received different types of services, four separate logistic regression analyses examined the relationship between the five risk factors and whether or not different types of services were received. Logistic regression models produce odds ratios for the independent variables. These odds reflect the increase or decrease in the likelihood of an outcome (e.g., receipt of a service) for every one-unit increase in the independent variables (e.g., risk factors). Each logistic regression analysis included one of the four types of services as the outcome variable: mental health, operational, juvenile justice, or social services. The logistic regression analyses were also calculated using the MPlus statistical package.

Results

Results of the first Poisson regression model for total services recommended revealed that youth history of substance use and family history of substance use were significantly related to the number of total services recommended (β = 1.41 and β = 1.46, p < 0.05). The second Poisson model for total services received revealed that family history of substance use (β = .90, p < 0.05) was significantly and positively related to the number of total services received, and there was a negative trend for history of out-of-home placements (β = −3.58, p = 0.07). Results of the multiple regression model for (square root of) dosage of total services as the outcome indicated that youth history of substance use was the only significant risk factor (β = 0.52, p < 0.05). The regression coefficients for both Poisson regression analyses and the multiple regression analysis are presented in Table 3.

Table 3 Results of two Poisson regression analyses—one including the outcomes of total services recommended and one including total services received—and one multiple regression analysis examining dosage
Full size table

Finally, four separate logistic regression analyses were employed to predict whether a type of service was received from the five risk factors. The first logistic regression model examined the dichotomous variable of mental health services received. For this analysis, there were trends for family history of mental illness (β = 1.12, p = 0.08) and history of out-of-home placements (β = −2.32, p = 0.07). The second logistic regression model included operational services received as the outcome and revealed that family history of mental illness was a significant predictor of whether or not youth received operational services (β = 1.54, p < 0.05). The third logistic regression analysis examined the outcome of juvenile justice services received. For this model, there were no significant predictors. The fourth and final logistic regression analysis focused on whether or not youth received social services, and this model also did not reveal any significant predictors.

Table 4 reports the parameter estimates and odds ratios from the logistic regression models examining the effects of the risk factors on whether or not youth received different services. An interpretation of the odds ratios reveals that youth with a family history of mental illness were nearly five times more likely to receive operational services (OR = 4.71) and three times more likely to receive mental health services (OR = 3.05) while controlling for all other variables in the model.

Table 4 Results of the four logistic regression analyses
Full size table

Discussion

The present study sought to determine whether previously identified risk factors are not only associated with negative outcomes in a system of care but also with service system variables, which have also been found to be related to youth outcomes within systems of care. The first set of analyses examined whether the presence of these risk factors was associated with the number of services recommended and received and the dosage of services received. The next group of analyses focused on which risk factors predicted whether or not a youth received different types of services, including mental health, operational, juvenile justice, and social services.

The results suggest that previously identified risk factors for negative outcomes in systems of care are differentially associated with service access. Specifically, youth who had histories of substance use and family histories of substance use had more services recommended than youth without these risk factors. In addition, youth whose caregiver reported a family history of substance use received more services, and youth who had a history of out-of-home placements received fewer total services. However, due to the low number of youth with out-of-home placements (5.9%) and the trend level effect, this result needs to be interpreted with caution. In addition, youth who had a history of substance use were significantly more likely to receive a higher dosage of services. Finally, a history of family mental illness was associated with receiving mental health and operational services, while youth history of out-of-home placements was related to not receiving mental health services.

Implications for Behavioral Health

Increased access to services is one of the goals of systems of care and has been found to be related to improved outcomes and reduction of symptoms for youth receiving services within a system of care. Walrath et al.10 found that risk factors had a stronger influence on deterioration at 6 months than lack of services received for children enrolled in systems of care. The present study extends these findings by revealing how risk factors influence service system variables.

Results of this study highlight the need to evaluate the referral process within systems of care to ensure that at-risk youth are identified and equitably provided appropriate prevention and intervention services. Consistent with previous research suggesting that substance use is associated with mental health symptoms,9 22% of this population of youth with severe emotional and behavioral challenges reported a history of substance use, and 33% of caregivers reported a family history of substance use. This finding is especially striking given that previous studies have suggested that these youth often have the greatest unmet needs for services.11 Therefore, the findings that youth enrolled in this system of care who had a personal and family substance use history had more services recommended, that youth with family substance use histories received more types of services, and that youth with personal substance use histories received a higher dosage of services suggest that this system of care is addressing, or attempting to address, the service gaps for youth with these risk factors. Future research should examine this further and identify which specific services are being recommended and received, as well as any barriers to receiving these services, in order to continue increases in access for this at-risk population.

A history of family mental illness was found to be significantly related to receiving different types of services, such as mental health and operational services. Previous research indicates that the rate of mental health problems in children of parents with a mental illness is twice as high as peers without this risk factor.22 Moreover, early onset of mental health problems in this population of children has been found to be associated with a range of negative outcomes.20,29 The results of the present study indicate that not only are children with mental illness in their family being identified in this system of care, but they are also being provided with services focused on mental health symptoms and family needs (i.e., family support and advocacy, emergency funds). Such services have been highlighted not only as effective treatments focused on existing symptoms but also important preventive interventions aimed at reducing or eliminating future mental health problems and providing needed family management in response to social and structural issues that are often encountered by these families.20,30

In contrast to these significant positive relationships between risk factors and service system variables, a history of suicide attempts was not significantly related to any of the service system variables, and a history of out-of-home placements was negatively related to the number of total services received and whether or not youth received mental health services. However, these findings must be interpreted with caution due to the small percentage of youth in the sample who were identified as having histories of a suicide attempt (11%) and out-of-home placements (6%).

The results of the present study need to be considered with some caution due to a number of potential limitations. First, there was no comparison group of youth receiving services outside of a system of care. Therefore, it is not possible to delineate if the results are specific for youth in systems of care or if they apply more generally to youth receiving any services or treatment. Second, the risk factor variables were provided by a parent/caregiver responding to single items on a questionnaire, with the exception of youth substance use history which was a composite of two separate items provided by the caregiver and the youth. In addition, the service variables, with the exception of service dosage, were provided by care coordinators and were not confirmed by caregivers or youth. The inclusion of multiple sources of information and several different items or indicators of risk factors would have provided more comprehensive and psychometrically sound data. Another limitation is the cross-sectional nature of the study. Perhaps an analysis of system variables over time would reveal new patterns; this is an important consideration for future research. Finally, the generalizability of these findings needs to be considered given that this sample had low percentages of youth with histories of out-of-home placements and suicide attempts and because data on services recommended and received were only available for a portion of the sample (85 out of 125).

This paper provides a preliminary understanding of the impact of risk factors on service access for families within a school-based system of care. The results are encouraging for systems of care since they indicate that prominent pre-referral risk factors are influencing service-level decisions and service delivery. The fact that youth who may be at a higher risk of future mental health problems are having more services recommended and are receiving more services is a positive sign for service system operations and provides more evidence that system of care principles, such as accessibility, individualized care, and family-focused services, are being implemented.6 Past research on youth and family risk factors has highlighted the need for integrated and coordinated prevention and intervention services.15,20 Continued research is needed to examine longitudinal outcomes and establish direct relationships between risk factors, reception of different types of services, and improvements in symptomatology and functioning over time. Moreover, such service system assessments should be conducted in other systems of care to identify differences and similarities between systems based on age and needs of children served, services offered, and service settings. Illuminating the ways in which service system components are influenced by youth and family risk factors and how they influence outcomes will enhance knowledge of service system operations in systems of care and will, subsequently, help to inform the design and refinement of service planning and implementation.