Introduction

Rehospitalization is considered to be an unfavorable outcome due to the cost of inpatient treatment and the stress and disruption that this acute treatment modality creates for youth and their families (Burns et al. 1999; Chung et al. 2008; James et al. 2010). However, as policies have driven down the use of hospitalization, primarily through reductions in lengths of stay, rehospitalizations have increased, creating concerns of a “revolving door” phenomenon (Chung et al. 2008). This has led to increased calls to understand the factors that may lead to rehospitalization among youth. The variables that have been studied in the rehospitalization literature are predominantly at the child level, mostly demographic (e.g., age, gender), clinical (e.g., diagnosis) and service utilization (e.g., length of stay, post-discharge services) variables. Very little research has explored the effect that hospitals might have on the rehospitalization of youth, which is the primary goal of the current study.

Rehospitalization as a Quality of Care Indicator

Recent trends in national healthcare policy now hold hospitals more accountable for patients’ readmission rates (Department of Health and Human Services 2011), however these policies have not been applied as frequently to psychiatric hospitalization. Instead, the more common view appears to be that psychiatric hospital outcomes should be confined to the episode of care (e.g., LOS, decline in psychiatric acuity on the unit) and that readmission rates are an outcome for which community providers should be accountable. For example, Lyons et al. (1997) suggest that rehospitalization should be understood as a reflection of the course of mental illness, a representation of general admission policies (e.g., threshold for hospitalization), and an indicator of the quality of community-based services. However, this view is not universal. Thakur (1998), for example, argued that the prevention of rehospitalization is not the responsibility of only community care providers, and suggests that the entire system of care (SOC), including inpatient providers, is charged with treating the patient and promoting community-based placement. Research documenting the relationship between factors related to the hospitalization itself and readmission, including the association between discharge preparedness and early readmission (Durbin et al. 2007) and patient-reported symptom improvement during admission and fewer subsequent readmissions (Byrne et al. 2010) provide some support for the claim that inpatient providers play a role in community tenure and readmission.

The differing views on the accountability of hospitals for readmission rates have not been submitted to much empirical scrutiny. It seems that the best and most direct test of these differing views would be to test whether hospitals and hospital systems naturalistically demonstrate significant variability in readmission rates. For example, if enough hospitals are studied in a system, and assuming that child-level variables can be adequately controlled, then research can test the hypothesis that hospitals vary in their readmission rates. If hospitals significantly vary in risk-adjusted readmission rates, then this suggests that hospitals can be understood to influence readmission rates and therefore may be considered partly accountable for this outcome.

Unfortunately, there is a dearth of research regarding the role of hospital provider on youth psychiatric rehospitalization. The majority of the extant literature has examined only one inpatient hospital and a number of the studies that do include more than one hospital provider do not consider hospital as a covariate in analyses (e.g., Romansky et al. 2003, Wickizer et al. 1999). Of the nineteen studies we found exploring predictors of rehospitalization in child and adolescent samples, only two examined the role of the hospital in rehospitalization. Foster (1999) examined three providers using data from the Fort Bragg study, but combined the two comparison sites. Overall, the comparison and demonstration sample participants varied in readmission rates but the obvious confound between service condition and the hospitals makes it difficult to draw any conclusions about the role of the hospital in readmission rates in this study. Fontanella (2008) compared three separate hospitals, in which hospital provider was included as a dummy variable in the analyses; the dummy-coded hospitals had a significant effect on rehospitalization. And, while Fontanella’s (2008) sample of three hospitals represented 72 % of private psychiatric beds in the state, many states use a much larger number of hospitals to meet the inpatient needs of their publically funded populations.

Despite the limited attention hospitals have received as predictors of readmission, a growing body of literature has begun to demonstrate that hospital providers can vary along key outcomes. For example, Rosenau and Linder (2003) conducted a review of seventeen studies comparing for-profit and not-for-profit hospitals on cost, access, and quality. The authors concluded that non-profit hospitals evidenced lower costs, more access or better quality compared to for-profit hospitals in 70 % of studies. The length of stay (LOS) literature has also accumulated evidence demonstrating variability across hospital providers. For example, Gifford and Foster (2008) found 51 % of variation in youth psychiatric inpatient LOS to be attributable to facility-level factors and only 4 % attributable to the time-invariant patient-level factors (e.g., gender, race, diagnosis) traditionally used to study LOS outcomes. Leon et al. (Leon et al. 2006) found significant variation across hospitals in LOS in a sample similar to the one used in the current study, Illinois children receiving inpatient services.

Child-Level Factors Associated with Psychiatric Rehospitalization of Youth

While variables associated with the hospital have received little attention in the readmission literature, child-level and service-related variables have received significant attention. It is critical to control for the effects of these variables in order to adequately study the role that hospitals might play in readmission.

Clinical characteristics studied in the rehospitalization literature include diagnosis (e.g., conduct disorder), risk variables (e.g., suicide risk), and co-morbidity (e.g., learning disabilities), and all have demonstrated strong associations with readmission (e.g., Arnold et al. 2003; Blader 2004; Romansky et al. 2003). However, disruptive behavior disorders such as oppositional defiant disorder and conduct disorder (American Psychiatric Association 2013) appear to have the strongest association with readmission in the literature (Blader 2004; Chung et al. 2008; Fite et al. 2008; Foster 1999).

With regard to demographic factors, the literature examining these variables is inconsistent and, at times contradictory. Some research suggests that younger youth are more likely to be rehospitalized compared to their older peers (Arnold et al. 2003; Bobier and Warwick 2005; Pavkov et al. 1997; Romansky et al. 2003). However, other studies suggest adolescents are at greater risk of rehospitalization (Fontanella 2008; Lapointe et al. 2010). The majority of literature does not report a significant relationship between gender or race and readmission to an inpatient facility (e.g., Blader 2004; Foster 1999; Romansky et al. 2003). The remaining research is contradictory, for example, Arnold et al. (2003) found Caucasian youth at greater risk of rehospitalization and Lapointe et al. (2010) suggest that African American youth have elevated rate of multiple hospitalizations. Finally, children in the child welfare system have been found to experience shorter post-discharge community tenures,, but research using this variable is limited (Burns et al. 2004).

Congregate care setting placement (e.g., residential treatment, group home, correction facility; Chung et al. 2008; Fontanella et al. 2010; Romansky et al. 2003; Stewart et al. 2014) has been shown to be associated with an increased risk of rehospitalization. Specifically, Romansky et al. (2003) found the rehospitalization rate to be over 25 % for youth placed in a congregate care setting post-discharge from inpatient services, compared to the 20 % rate of youth placed in foster homes and the 13 % rate of youth living independently or with a family member (parent or relative). Chung et al. (2008) suggest that the higher rate of rehospitalization found in youth placed in congregate care settings may be due to the lower threshold for psychiatric hospitalization of the staff members.

Community variables have also been employed to predict time to rehospitalization and include community socioeconomic status (SES) and post-discharge community services. Pavkov et al. (1997) found that discharge to a low socioeconomic community was associated with rehospitalization risk. Studies assessing the impact of aftercare services on readmission have been contradictory, with some evidence that higher levels of post-discharge community-based services are associated with lower readmission rates (Romansky et al. 2003), and other studies suggesting that aftercare is associated with increased likelihood of (Carlisle et al. 2012) and more rapid rehospitalization (Fontanella 2008).

In terms of service use predictors, index hospital LOS and number of prior hospitalizations are the two most common predictors of rehospitalization in the literature. The adoption of managed care and the documented reductions in LOS for inpatient psychiatric hospitalization of youth (Case et al. 2007) has resulted in increased attention to the relationship between LOS and rehospitalization and the possibility that decreased LOS could lead to higher overall inpatient service use when factoring in overall utilization across episodes of care (e.g., the “revolving door phenomenon”). Wickizer et al. (1999) found application of a utilization management program, which decreased LOS, to result in decreased inpatient “resource consumption” and an increased risk of readmission. Figueroa et al. (2004) also found an inverse relationship between LOS and readmission, with slight decreases in LOS associated with significant increases in risk of readmission in a mixed population of adults and youth. Recently, Yampolskaya et al. (2013) also found an inverse relationship between LOS and risk of readmission. Number of prior hospitalizations has shown similar success as a predictor of readmission across populations and time (e.g., Bickman et al. 1996; Chung et al. 2008).

The current study examines days to rehospitalization for a sample of youth across 29 hospitals in Illinois from January 1st, 2005 through May 8th 2006. Utilization was reimbursed through Medicaid, and youth were either in the child welfare (Department of Children and Family Services; DCFS) or Medicaid only (Department of Human Services; DHS) systems. The current study uses a multi-level modeling survival analysis strategy to account for the fact that days to rehospitalization are nested within hospital and to provide an estimate of the relative influence of hospitals versus child-level factors on time to rehospitalization. Further, a wide range of child-level variables are used to predict utilization. These include clinical, functioning, juvenile justice status, and caregiver items from the children’s severity of psychiatric illness (CSPI) scale, race/ethnicity, LOS (index hospitalization), and placement status. We hypothesize that, consistent with prior research in the readmission and broader mental health care literature, that hospitals will vary significantly in time to readmission.

Method

Setting

The current study was conducted through the Screening, Assessment, and Supportive Services (SASS) program of Illinois. The SASS program was implemented in 1992 to provide crisis assessment and treatment services to children in protective custody who are referred for or at risk of hospitalization. In recognition that children’s mental health services in Illinois were in need of improvement, the Children’s Mental Health Act of 2003 was signed into law. As part of this act, all children who are Medicaid eligible are required to have pre-admission screenings prior to entering the psychiatric hospital in order to insure that referrals meet appropriate criteria for a hospitalization. Therefore, in 2005, SASS’ services were expanded to include all publicly funded children in Illinois.

Referrals are made to SASS workers when a child who receives Medicaid in Illinois demonstrates a risk of self-harm or injury to others. Telephone referrals can be made by relevant parties such as DCFS personnel (e.g., caseworkers) or clinicians from a hospital to which a child has been presented for admission. For eligible children, pre-hospitalization screenings and crisis intervention services are administered within 4 h of the referral. Screening interviews take place with the child and caregiver and are intended to assess whether the child has reached a level of risk that warrants hospitalization.

Sample

The participants were Medicaid-insured, and primarily children and adolescents ranging in age from four to 24 years old from Illinois (n = 1443) who were screened by SASS and hospitalized in calendar year 2005 through May 8th, 2006. Hospitalizations were funded by Medicaid through the Illinois Department of Public Aid (DPA). All hospitals are paid on a per diem basis based on contracts with DPA. Twenty-nine hospitals were selected from a total of 33 hospitals based on the number of children represented who received treatment at the facilities during the study period. To maximize power at the hospital level, sample sizes greater than or equal to ten were selected, leaving out four hospitals.

Procedure

SASS workers completed the children’s severity of psychiatric illness (CSPI; Lyons 1997) and a demographics form on all screening cases. Completion of the demographics and CSPI forms are a contractual obligation and tied to auditing and reimbursement for the SASS agencies. Workers are instructed to complete all paperwork prior to the end of their shifts. The SASS workers also completed monthly reports on all children who continue to require hospitalization. The Institutional Review Boards at Northwestern University and Loyola University Chicago approved the study.

SASS workers are masters’ level providers (Qualified Mental Health Professionals; QMHP) or Mental Health Professionals directly supervised by a QMHP. To be certified to complete the CSPI, SASS workers must participate in a CSPI training and complete a reliability vignette with an accuracy (percent correct) of 85 % or above.

Measures

Dependent Variable-Days to Readmission

Rehospitalization was calculated as occurring when a child’s DCFS or Medicaid ID reappeared in the dataset, indicating a second hospitalization. Days to rehospitalization (right-censored) was computed as the number of days between the discharge date of the first hospitalization and the intake date of the rehospitalization. For children who did not rehospitalize in the study period, days were determined by the number of days between the primary hospitalization discharge date and the end of the study, May 8, 2006.

Demographics

Using the demographics and monthly reporting forms completed by SASS workers allowed for the determination of the child’s age, race/ethnicity, custody and placement status prior to intake (e.g., child welfare involved, residential treatment placement), and LOS (index hospitalization).

Children’s Severity of Psychiatric Illness (CSPI)

The CSPI is a measure of clinical and environmental factors developed from focus groups and the literature (Lyons 1997). The CSPI provides severity ratings on a four-point scale (rating of 0–3). For instance, for Suicide Risk, a rating of “0” would be given to indicate a child with no evidence of past or current suicidal ideation, while a rating of “3” would be used to indicate a child with ideation and intent within the past 24 h (Lyons 1997). The CSPI provides information regarding the child’s risk behaviors, behavioral/emotional symptoms, functioning problems, juvenile justice status, child protection status, and caregiver needs and strengths. The CSPI has been found to be valid for decision support to hospitals (Leon et al. 1999). Inter-rater reliability has been found to vary between .70 and .89 (Lyons et al. 2002).

The CSPI is frequently used at the item level in the literature. Therefore, in an effort to condense the items, the CSPI was factor analyzed using a principal components analysis extraction with direct oblimin rotation (Gorsuch 1974; Preacher and MacCallum 2003). Items with loadings of .30 or higher were assigned to their related scales. Items were allowed to load on multiple factors if absolute values of their factor loadings were greater than .50 on multiple scales. Examination of the scree plot indicated the presence of four interpretable factors, labeled as follows along with the individual CSPI items comprising the scale: (1) externalizing behavior, consisting of danger to others, judgment, fire setting, social behavior, impulsivity/hyperactivity, oppositional behavior, conduct disturbance, anger control, school functioning and peer functioning; (2) internalizing behavior, consisting of suicide risk, self-mutilation, depression, anxiety and adjustment to trauma; (3) juvenile justice involvement, consisting of juvenile justice status, juvenile justice community safety, juvenile justice delinquency, and community functioning; (4) caregiver needs and strengths, consisting of caregiver health, caregiver supervision, caregiver involvement, caregiver social resources and caregiver residential stability (see Table 2). The alpha statistics for Externalizing Behavior, Internalizing Behavior, Juvenile Justice Involvement, and Caregiver Needs and Strengths, respectively were: .84, .64, .77 and .78.

Missing Data and Analytic Procedure

The data for this study were taken from an original sample of 1466. However, 23 cases had missing data and were dropped from the analyses.

This study integrated survival analysis and multilevel modeling by conducting person-level and a multilevel continuous time survival analysis to model return to inpatient psychiatric hospitalization following hospital discharge. Survival analysis is an event history approach that allows for an evaluation of when a given event occurs (Singer and Willett 2003). The event for the present study includes the rehospitalization, or return to treatment, following inpatient treatment. The time to the event is measured in days, which was calculated by subtracting the date of original discharge from the date of the child’s return to treatment. In the present study, children may have been hospitalized multiple times before the present observation period, but we are focusing on the first return to treatment during the current observation period.

To predict time to events, a multilevel Cox proportional hazards model was employed using Mplus Version 7.1 (Muthén and Muthén 2013). This analytic approach originally developed by Cox (1972), assumes a partial likelihood approach in which survival time is regressed on covariates that are hypothesized to be associated with the survival distribution. In the present case, a linear relationship is proposed and this model is semi-parametric because the baseline hazard can take any form and the covariates are entered into the model linearly. This model allows for no assumptions of the underlying form of the baseline hazard model, which allows for an accurate extraction of the estimate of the baseline hazard. In the child-level model, the person-level predictors of age, LOS (index hospitalization), dummy coded race/ethnicity values (African–American and Caucasian), dummy coded residential placement, CSPI composite externalizing behaviors, internalizing behaviors and caregiver needs and strengths were all included as simultaneous predictors of the continuous variable time to return to treatment.

The proposed multilevel model is an extension of traditional Cox regression analysis, which uses non-parametric Cox proportional hazards model (Cox 1972) that incorporates single-level predictors and a random intercept in the model that allows for variability at the cluster level (i.e. hospitals). While there are no hospital level predictors, this form of data analysis accounts for the individual characteristics (single-level predictors; e.g. externalizing behaviors) and context (clustering; i.e. hospitals) in a single model. In this model, the days to return variable was regressed on level-1 predictors and, in contrast to the child-level model, the variance of time to return to treatment between hospitals was freely estimated. Using this analytic approach, the model allows for estimation of between hospital variation while taking into account for single-level predictors to detect differences in survival rates between hospitals. In addition, it should be noted that in general linear models, there is an assumption that cases are random samples from the population and the observed scores for the dependent variable, which in this case is time to return to treatment, are independent of one another. However, in multilevel models this assumption is violated as individuals within the same group (i.e. hospitals) are more similar than those in different groups. While the groups are independent from one another, the observations within the groups may share characteristics and therefore are not independent. Multilevel modeling takes into account the similarities within the groups by the intraclass correlation, which assumes observations within the groups are more similar than the observations from between groups.

Inter-correlations among the four CSPI composite variables were small (i.e., below .30) except for one. The correlation between Externalizing Behavior and Juvenile Justice Involvement was .46, p < .001. Therefore, to avoid issues with multi-colinearity, Juvenile Justice Involvement was not included in the survival analyses. Further, there was considerable overlap between DCFS custody status and residential treatment placement; 77.6 % of the sample who were placed in a residential treatment setting were also in DCFS custody. Due to the research base suggesting a significant association between residential treatment setting placement and greater likelihood of return to the hospital, the dummy-coded variable DCFS custody was dropped from the survival analyses.

Results

Descriptive Statistics and Univariate Comparisons

Demographic data are included in Table 1. The final sample consisted of 1443 children and adolescents with ages ranging from 4.35 to 24.20 years old (M = 14.33, SD = 3.19) from 29 hospitals in Illinois. The mean LOS for each individual was 11.18 days (SD = 9.80). Of the 1443 individuals in the sample, 160 (11.1 %) were re-admitted to a psychiatric hospital with an average of 35.01 days (SD = 31.87) to return to treatment. Comparing readmission rates across the sample hospitals, these rates ranged from zero to 23 %. However, a χ 2 test comparing rates across the hospitals was not significant, χ 2 (28, n = 1443) = 33.08, p = .23. Further, among the 10 hospitals with five or more readmissions, there were no significant differences in the days to readmission, F(9, 102) = 0.77, p = .65. Among the five hospitals with more than 10 readmissions, once again the number of days until readmission was not significant across the hospitals, F(4, 78) = 0.74, p = .57 (Table 2).

Table 1 Sample demographic, CSPI, living arrangement, custody status, and length of stay descriptives (n = 1443)
Full size table
Table 2 CSPI Items and factor loadings comprising externalizing behavior, internalizing behavior, juvenile justice involvement and caregiver needs and strengths scales (n = 1443)
Full size table

Child-Level Model

In this model, the person-level variables were used as simultaneous predictors of the continuous variable time to return to treatment The results of the analyses indicated that the Externalizing Behavior, b = 0.031, SE = 0.012, p = 0.013, and residential placement, b = 0.744, SE = 0.337, p = 0.027, variables were associated with faster return to hospitalization. Age, race/ethnicity, index hospitalization LOS, Internalizing Behaviors, and Caregiver Needs and Strengths were not significant predictors of time to return to treatment. See Table 3 for parameter estimates for both the child-level and multilevel models. Visual inspection of the survival curves for the residential placement variable and a data-binned Externalizing Behavior variable consisting of three levels (low, moderate, high) revealed that none of the survival lines overlapped as a function of time, indicating that the proportionality assumption was met.

Table 3 Predictors of time to return to treatment for both child-level and hospital level cox proportional hazards models (n = 1443)
Full size table

Multilevel Survival Analysis

To evaluate the influence of hospital on return to treatment, a multilevel survival model was tested using non-parametric Cox Proportional Hazards model (Cox 1972). In this model, the days to return variable was regressed on level-1 predictors and, in contrast to the Child-Level Model, the variance of time to return to treatment between hospitals was freely estimated. The results indicate the variance accounted for by hospital was not significant [var(hospital) = 0.042, t = 1.202, p = 0.229], suggesting that survival rates did not differ between hospitals when taking into account the person-level predictors.

In order to evaluate the goodness of fit to the data for both models, information theory goodness of fit measures were reviewed. These measures include the AIC, BIC, and ABIC, which are applicable for model comparison for non-nested models and when maximum likelihood estimation is used. While there is no formal cutoff for these measures, models with the lowest information theory goodness of fit measures are deemed optimal. In the present case, the AIC, BIC, and ABIC values for the child-level model are lower than the observed values for the multilevel model suggesting less deviation between the observed and predicted covariance matrices and therefore was selected as the final model.

Discussion

The current study explored readmission rates across 29 hospitals serving Medicaid-insured children in one state. A multi-level survival analysis approach was applied to the data to examine whether hospitals significantly vary in their readmission rates after controlling for child-level variables. The results found that the 29 hospitals did not vary significantly in readmission rates after controlling for child-level variables. The only child-level variables associated with readmission were externalizing behaviors and placement in residential treatment.

These results support the perspective that hospital outcomes are best operationalized using variables tied more directly to the inpatient episode (e.g., LOS, reductions in acuity) and Lyons’ (1997) view that accountability for readmission rates should be connected to factors associated with experiences the child has in the post-discharge community (e.g., services received from community providers). Much of the inpatient outcomes literature has focused on variables associated with only the episode of care; the primary example is LOS. LOS is an outcome which by its very definition encompasses the episode of care and, unlike readmission, does not include variables for which the hospital has little control, such as the quality of post-discharge services in the community, the school or residential treatment environment, or general community conditions. The LOS literature has consistently and robustly found significant variation in LOS at the hospital level (e.g., Leon et al. 2006; Gifford and Foster 2008). Results such as this imply that hospital practices might vary in such a way that leads to differences in severity-adjusted LOS rates. A better understanding of these hospital practices could then lead to the development of best practice standards aimed at reducing LOS. The same implications would not be supported by the results of the current study as it relates to readmission.

Thakur (1998), in disagreeing with Lyons (1997), argued that the prevention of rehospitalization is not only the responsibility of community care providers but of the entire SOC, including inpatient providers. From a larger, systemic perspective, the results in this study do not necessarily argue against this position. In fact, the results here may be representative of a lack of integration between the inpatient and community service settings that, despite the System of Care model, has still not become a reality (Lyons 2004). The lack of clinical integration between inpatient and community services may be so pervasive that despite potential variation in the quality of inpatient services (e.g., post-discharge planning) across inpatient settings, this variation does not translate into variations in time to rehospitalization because of larger systemic mores preventing continuity of care.

The results here are contradictory to some of the extant literature on readmission. For example, in the only other study to adequately explore the role of hospitals in readmission rates, Fontanella (2008) found significant variation in readmission rates across the three sample hospitals. And although the sample of hospitals was small, Fontanella’s (2008) study nonetheless encompassed the majority of private beds in the state, suggesting strong generalizability within the state examined. Further, Fontanella’s (2008) findings are more consistent with the broader hospital effects literature, namely that the hospital can have a significant impact on outcomes. One explanation for this discrepancy may have to do with how access to inpatient care is managed in Illinois’ public sector. Illinois’ Medicaid funded inpatient episodes must first be approved by independent mobile crisis (SASS) workers. Prior research has found that the SASS program has been successful in reducing unnecessary hospitalizations across the state (Leon 2008). Therefore, it is possible that in a highly managed inpatient utilization environment, hospitals are less likely to vary in readmission rates. However, it is also important to note that prior research in Illinois has found that even with the implementation of the SASS program, hospitals vary significantly in LOS (Leon et al. 2006).

An advantage to this study is that the 29 hospitals studied in this sample represent the largest hospitals in the state serving Medicaid children in this time period. Studying a larger sample of hospitals that more closely approximate the provider base in one state provides a more accurate set of policy implications. As the health care field in general engages in an ongoing effort to define the parameters of hospital provider accountability (DHHS 2011), it is important for the psychiatric hospital service sector to contribute to the research base, specifically the role that hospitals play in psychiatric readmissions compared to other health conditions. Without such a research base, the psychiatric hospital system may be required to adhere to accountability standards that are mismatched with the role that psychiatric hospital services plays in people’s treatment.

The current study suggests that the variables in this sample associated with readmission are externalizing behaviors and placement in a residential facility. These findings are consistent with the prior literature. Externalizing behaviors, represented by diagnosis or standardized measures, have consistently been associated with readmission in the literature (e.g. Blader 2004; Chung et al. 2008; Fite et al. 2008; Foster 1999). It remains unclear whether this and other similar findings in the literature points to the challenging course and management of externalizing behavior in the community or a general and persistent mismatch of resources for youth with externalizing behavior. This represents a critical goal for future research.

The finding that residential treatment facility was associated with greater likelihood of readmission is consistent with most (e.g., Romansky et al. 2003), but not all (Fontanella 2008) previous research. This finding is disheartening because it suggests that something about being placed in congregate care—the stress of living in a congregate setting, lower thresholds for referral—is associated with readmission to the hospital, after controlling for clinical characteristics. The findings here may be consistent with a broader set of findings in the residential treatment outcomes literature, some of which has suggested that this level of care may at times under-perform compared to community-based alternatives. Studies have shown that placement in outpatient facilities may be more effective and cost effective than more intensive services such as residential care (Hermann 1997). Research has also shown that there are no differences in the reduction in problem behaviors between children in treatment foster care and residential treatment centers, and children in treatment foster care actually function better in less restrictive placements when discharged (Barth 2002; Bates et al. 1997).

Limitations

This study has several limitations. First, as is often the case with naturalistic study designs, collecting data from workers in the field leads to challenges to internal validity. For example, SASS workers most likely vary in the environments in which they complete the CSPI (e.g., in the field versus office), and unstandardized contexts for measure completion can compromise validity. In terms of generalizability, this study takes place in one state with a unique service environment and demographics, suggesting that our results are generalizable only to Illinois and its unique service environment. However, it is also important to note that not every hospital in the overall Illinois sample was used due to sample size. Out of an original sample of 33 hospitals, four were omitted for treating fewer than 10 children.

Our use of the multilevel survival analysis approach to account for child and hospital variability in our hypotheses is an important avenue for research. However, our results should be interpreted with caution due to the small sample size at the hospital level (n = 29). Consequently, our ability to test for between-hospital effects was underpowered in comparison to our ability to test for child level outcomes, which has implications for the interpretation of results. More specifically, Maas and Hox (2005) and Paccagnella (2011) provide evidence that for small between-level sample sizes (n < 50), estimation of standard errors for the between-level may be inaccurate leading to inaccurate interpretations of results. However, it should be noted that while the between level standard errors provide evidence of bias, the individual level estimates did not indicate such bias. In sum, these simulation studies highlight the need for caution in interpreting our lack of evidence for hospital level effects as parameter estimates may have been biased as a function of sample size and emphasizes the need for greater exploration in the role of hospitals on return to treatment outcomes (Table 4).

Table 4 Fit statistics for child level and multilevel models (n = 1443)
Full size table

We were not able to measure all of the variables found in previous research to predict readmission. For example, number of prior hospitalizations is one of the more robust variables in the literature predicting readmission. However, we were not able to measure number of prior admissions because the study window was a year and a half, and we were not able to collect data on admissions before this period. Another common variable in the readmission literature is post-discharge services and we were also unable to measure these services in this study. Finally, we were not able to collect data on hospital characteristics (e.g., state, private, non-profit, bed capacity).

Conclusions

Overall, these results suggest that hospitals may not vary significantly in time to readmission for children and adolescents receiving Medicaid-insured psychiatric services. These findings suggest either that (a) it may be inappropriate to hold hospitals accountable for time to rehospitalization, consistent with Lyons’ (1997) view that too much happens in the post-discharge environment for which hospitals have no power to control or; (b) that both the hospital and the community can and should be accountable, but that the lack of integration and continuity between these two treatment worlds is so pervasive that little of what hospitals do or recommend make it into community practice settings. Support for this latter possibility could be derived from future research measuring hospital-community continuity of care as a predictor of time to rehospitalization, an area of research in need of greater development (Adair et al. 2003). Given the limited work that has been done using hospital as a predictor of time to rehospitalization in the child and adolescent literature, future work should continue to measure hospital as a nested variable in multi-level models to support or contradict the results here.