Introduction

The purpose of assessment within community mental health is to obtain an accurate and complete understanding of a youth’s behavioral and emotional functioning in order to make informed decisions about how to help the youth succeed (Salvia et al. 2012). Historically, assessment within community mental health has focused on youths’ deficits and pathologies (Epstein 1999). In a deficit-based approach, clinicians use assessment to identify problem behaviors, diagnose mental health disorders, plan treatments, and measure treatment outcomes (Whitcomb and Merrell 2012). Within the last two decades, however, there has been a shift to the identification of emotional and behavioral strengths (Epstein et al. 2001; Harniss and Epstein 2005).

Strength-based assessment is conceptualized as the measurement of the emotional and behavioral skills and competencies that increase youths’ abilities to promote positive relationships with others, cope with adversity and stress, and enhance a sense of personal achievement (Epstein 2004). A strength-based assessment framework focuses on the measurement of youths’ competencies in order to obtain a holistic and comprehensive understanding of the youth (Epstein et al. 2001; Oliver et al. 2014). By measuring youths’ strengths, clinicians (a) can better develop, implement, and monitor interventions, (b) can promote optimism and a positive outlook on the potential benefit of treatment, and (c) are more likely to engage youth and families in treatment (Merrell et al. 2011; Tedeschi and Kilmer 2005).

Presently, there are a number of instruments available to assess the strengths of youth and within community mental health, such as the Behavioral and Emotional Rating Scale (Epstein 2004), the Child and Adolescent Needs and Strengths scale (Lyons 1999), the Strengths and Difficulties Questionnaire (Goodman 1997), and Devereux Student Strengths Assessment (LeBuffe et al. 2009). Despite the availability of multiple measures for assessing youths’ strengths, not all have well-established psychometric properties for youth within community mental health settings.

One of the most widely used scales for measuring youths’ strengths is the Behavioral and Emotional Rating Scale-Second Edition (BERS-2), a nationally normed and standardized assessment that has parallel forms for caregivers, youth, and teachers. The BERS-2 includes 52 items that comprise the subscales of Interpersonal Strengths (ability to interact with others), Intrapersonal Strengths (youths’ perception of his/her accomplishments), Family Involvement (relationships with family), School Functioning (competence in school), and Affective Strengths (ability to give and receive affection). The subscales combine to produce the Total Strength Index, which is an indicator of the youth’s overall strengths. Several studies support the reliability, validity, and factor structure of the BERS-2 in schools (Benner et al. 2008; Mooney et al. 2005) and residential settings (Duppong Hurley et al. 2014).

The psychometric properties of the BERS-2 are well-established in previous studies, although they have primarily included youth within schools settings. The one study that evaluated the convergent validity of the BERS-2 with youth in community mental health settings demonstrated evidence of moderate, negative correlations with a measure of functional impairment (Walrath et al. 2004). However, these researchers did not include other measures of emotional and behavioral functioning. To establish the validity of a scale such as the BERS-2 in a new setting, it is important to accumulate evidence that supports its psychometric properties (Messick 1995). Furthermore, test guidelines from professional organizations indicate that it is essential to re-establish the psychometric qualities of an assessment when it is used with a population who were not included in the initial standardization sample. Therefore, it is necessary to continue building evidence of the convergent validity of the BERS-2 for youth served in community mental health settings.

The purpose of this study was to evaluate the convergent validity of the BERS-2 with the Child Behavior Checklist (CBCL; Achenbach and Rescorla 2001) and the Columbia Impairment Scale (CIS; Bird et al. 1993), two well-known and widely-used assessments of youths’ behavioral and emotional functioning. It was expected that most of the BERS-2 subscale scores would be moderately and negatively correlated with the CBCL and CIS. However, given the similarity and close alignment of the construct measured, it was expected that the correlation between the school functioning scales of the BERS-2 and the CBCL would be positive and large.

Method

Participants

Participants eligible for inclusion were selected from all of the children and caregivers involved in the longitudinal study of the national evaluation of the CMHI (N = 11,518). The specific criteria for inclusion in the current study were: (1) youth between 6 and 18 years of age, (2) valid demographic data at baseline, and (3) valid assessment data at baseline for all of the measures used in the present study. Using these criteria, the sample included 7487 youth who were predominantly male (64.8 %; n = 4855). The racial/ethnic composition of the participants was diverse, as 44.1 % were Caucasian (n = 3301), 26.7 % were African-American (n = 1997), 17.4 % were Hispanic (n = 1300), 4.2 % were Native American (n = 318), 5.4 % were multiracial (n = 405), and 2.1 % were Asian/Pacific Islander (n = 156). The average age of youth was 12.17 years (SD = 3.13) and 47.6 % (n = 3564) had an Individualized Education Program (IEP). According to caregiver report, 51.6 % (n = 3863) of participants were living below the federal poverty line, 15.4 % lived at or around the poverty line (n = 1151), and 25.9 % (n = 1938) were living above the poverty line.

In order to examine differences between the current sample (N = 7487) and the excluded cases from the larger dataset (N = 4031), characteristics of the current sample were compared to the larger dataset on the demographic variables using Chi square and t tests (see Table 1). Effect sizes were calculated to express the magnitude of differences between the current sample and the larger dataset. There were minor, but statistically significant differences between this sample and the larger sample on gender and race/ethnicity. Treating the larger dataset as the reference group, relative risk ratios (RRs) were computed for each statistically significant difference. RRs indicate the increase in the rate of prevalence in the sample compared to the larger dataset. For instance, a RR of 1.07 for males would indicate the prevalence is 7 % higher in the sample than in the larger dataset, and a RR of 0.90 for Hispanic youth would indicate that the prevalence is 10 % lower in the sample than in the larger dataset.

Table 1 Demographic characteristics of participants
Full size table

Procedures

The Comprehensive Community Mental Health Services for Children and Their Families Program, also known as the Children’s Mental Health Initiative (CMHI), is a cooperative agreement program that is administered by the Substance Abuse and Mental Health Services Administration (SAMHSA), within the Department of Health and Human Services. The CMHI provides funds to community organizations to develop and implement mental health services for children from birth to age 21, with an overarching goal of promoting the mental health of children and adolescents. A secondary data analysis of the CMHI data was conducted for the present study. Communities included in this study represented three phases of program funding: Phase IV communities were initially funded between 2002 and 2004, Phase V communities were initially funded between 2005 and 2006, and Phase VI communities received initial funding in 2008. At the end of Fiscal Year 2009, a total of 97 communities across 45 U.S. states, districts, and territories had been funded. Within each Phase, recruitment of participants was ongoing with the majority of data used in the current study (84.7 %) collected between 2005 and 2010. Data collection for each community included structured interviews with parents/caregivers of children of all ages as well as with youth 11 years and older, which included the administration of several rating scales at intake (baseline) and subsequent 6-month follow-up periods [Time 1 (6 months), Time 2 (12 months), Time 3 (18 months), and Time 4 (24 months)]. For the purposes of the present study, only data collected at baseline were used. Detailed information about the protocol and data collection procedures is available elsewhere (Center for Mental Health Services 2010).

Measures

The Behavioral and Emotional Rating Scale-2 (BERS-2; Epstein 2004) was administered to caregivers at baseline to measure youths’ emotional and behavioral strengths. It is a 52-item scale in which raters judge youths’ competencies on a 4-point Likert-type scale from 0 to 3 (0 = not at all like, 1 = not much like, 2 = like, 3 = very much like). The BERS-2 produces scaled scores (M = 10, SD = 3) for the five subscales (Interpersonal Strengths, Family Involvement, Intrapersonal Strengths, School Functioning, and Affective Strengths) that comprise a Total Strength Index, which is reported as a standard score (M = 100, SD = 15). Scores for the subscales and the Total Strength Index were used in the current study. The test–retest, internal consistency, convergent validity, and content validity are well established for youth within school settings (Epstein 2004).

The Child Behavior Checklist (CBCL; Achenbach and Rescorla 2001) is a norm-referenced, standardized measure of the emotional and behavioral problems of children ages 6–18 that was administered to caregivers at baseline. The CBCL consists of 113 items with ratings on a 3-point Likert-like scale from 0 (not true) to 2 (very true or often true). Responses are combined to yield scores on the Syndrome Scales and Competence Scales. The Syndrome Scales include nine narrowband measures of functioning that are combined to produce a broad Total Problem Behavior T-score (M = 50, SD = 10). Also calculated are T-scores for Internalizing Problems (i.e., three Syndromes Scales that include measures of inwardly directed problems) and Externalizing Problems (i.e., two Syndrome Scales that measure conflicts with others and with expectations of the youth). Higher scores are indicative of greater problem behavior. Within the Competence Scales, the School Competence scale was utilized for the present study, which includes ratings of the youth’s performance in academic subjects, grade repetition, receipt of remedial services, and other school problems. Scores on the Competence Scales are reported as T-scores, with higher scores indicating greater competence.

The Columbia Impairment Scale (CIS; Bird et al. 1993) is a 13-item scale measuring youths’ functional impairment in the areas of work/school, interpersonal relations, use of leisure time, and other broad areas of functioning. Respondents rate the degree to which items in each area of functioning are problematic for the youth on a 5-point Likert type scale from 0 (no problem) to 4 (very bad problem). Responses to items are summed to produce an overall impairment score that ranges from 0 to 52. Higher scores suggest greater impairment, with scores >15 indicative of clinical impairment in the youth. In the present study, scores from the caregiver form only were used. The psychometric properties are well established for the caregiver version of the CIS (Bird et al. 1993, 1996).

Data Analysis

Pearson product-moment correlation coefficients were computed between BERS-2 Total Strength Index (standard score) and subscales (scaled scores) and the (1) CBCL Internalizing Problems T-score score, (2) CBCL Externalizing Problems T-score score, (3) CBCL Total Problems T-score, (4) CBCL School Competence T-score, and (5) CIS Total score. The magnitude of associations was evaluated using guidelines proposed by Cohen (1988) where correlations <.29 are considered small, correlations between .30 and .49 are considered moderate, correlations between .50 and .70 are considered large, and correlations >.70 are considered very large. Given the large sample size, the focus of the interpretation was on the direction and magnitude of the estimates, but statistical significance was also evaluated at the .001 alpha level.

Results

The concurrent correlation coefficients between the BERS-2 scores and the scores from the CBCL and CIS are detailed in Table 2. Correlations between BERS-2 scores and the CBCL Internalizing Problems scores represent small to moderate levels of negative covariation with estimates ranging from −.23 to −.44. The associations between the BERS-2 scores and the CBCL Externalizing Problems scores were negative and moderate to large in magnitude, with coefficients ranging from −.34 to −.69. The relation between the CBCL Total Problems scores and the BERS scores were also negative and moderate to large in magnitude (r = −.34 to −.60). Correlations between the BERS-2 scores and the CBCL School Competence scores were positive and ranged from small to large (r = .13–.53). Notably, the estimate between the BERS-2 School Functioning score and CBCL School Competence score had the largest magnitude, as expected (r = .53). Finally, correlations between the CIS Total score and the BERS-2 scores were negative and moderate to large, ranging from −.44 to −.66. All correlations were statistically significant at the .001 alpha level.

Table 2 Concurrent correlations between the BERS-2 and the CBCL and CIS
Full size table

Discussion

Within community mental health, assessment of youths’ emotional and behavioral functioning is often conducted from a deficit-based approach. More recently, there has been a shift to also identifying strengths, as they can provide a holistic understanding of youths’ functioning, which can better engage families and inform treatment planning. The BERS-2 is a standardized, norm-referenced measure of youths’ behavioral and emotional strengths that yields scores that have well-established psychometric properties in school and mental health settings. Given that previous studies have focused primarily on youth within school settings, the purpose of the present study was to evaluate the convergent validity of the scores from the caregiver version of the BERS-2 with measures of behavioral and emotional problems, using a large national and diverse sample of youth referred for community mental health services. It was hypothesized that (a) most of the BERS-2 subscale scores would be moderately negatively correlated with scores from the CBCL and CIS, and (b) there would be a large, positive relationship between the score for the BERS-2 School Functioning subscale and score for the CBCL School Competence subscale.

Overall, the findings supported both hypotheses, providing evidence for the convergent validity of the BERS-2 scores with CBCL and CIS scores for youth receiving services within community mental health settings. That is, BERS-2 subscale scores demonstrated moderate to large negative associations with the Internalizing Problems, Externalizing Problems, and Total Problems scores of the CBCL. Although correlations were generally larger between the BERS-2 subscale scores and the Externalizing Problems score than with the Internalizing Problems score, associations were greater between the subscales that measured similar constructs. More specifically, the largest correlation was between the BERS-2 Interpersonal Strengths score, which measures social interaction skills, and the CBCL Externalizing Problems score. Additionally, the CBCL Internalizing Problems score demonstrated the strongest relation with the BERS-2 Intrapersonal Strengths score, which measures youths’ insight on their competencies. Furthermore, the association between the Total Strength Index and the Total scores of the CBCL and CIS were negative and large. Given the magnitude and direction of the correlations with measures of emotional/behavioral functioning and functional impairment, results suggest that the BERS-2 scores measure distinct, but similar constructs as the CBCL and CIS. Finally, the association between the School Competence score of the CBCL and the School Functioning score of the BERS-2 was moderate and positive, indicating that the two subscales measure related constructs.

Findings are consistent with previous studies evaluating the convergent validity of the BERS-2 scores with youth in educational, residential, and mental health settings. In the present study, the moderate to strong associations between the BERS-2 scores and measures of externalizing problems were similar to results from research with youth in school settings (Benner et al. 2008; Mooney et al. 2005). However, associations between measures of internalizing problems and the BERS-2 were all significant in this study and were larger than in prior school-based research when evaluating the point estimates of the correlations. Furthermore, when compared with results of research with youth in residential settings, point estimates from the present study were generally larger in magnitude (Duppong Hurley et al. 2014). This could potentially be due to the fact that youth in the current study may have less severe emotional and behavioral problems than youth receiving residential care. Finally, correlations between BERS-2 and CIS scores were generally large in the current study, but in previous research with youth in community mental health settings, associations between BERS-2 scores and a different measure of functional impairment (i.e., Child and Adolescent Functional Impairment Scale) were only moderate (Walrath et al. 2004). Although the findings are similar, the larger correlations observed in the current study may be due to the use of different measures of functional impairment.

Implications for Practice

Findings from the current study support the use of the BERS-2 in community mental health settings to provide a comprehensive understanding of the youth. Advantages of the BERS-2 include its large research base and its normed referenced and standardized scores. Given these benefits and the adequate convergent validity of BERS-2 scores with measures of behavioral and emotional functioning and functional impairment, results support its use in community mental health settings to (a) develop a profile of youths’ emotional and behavioral strengths, (b) plan an appropriate treatment, (c) monitor youths’ response to the treatment, and (d) evaluate treatment outcomes.

Limitations

Although present findings have important implications for the assessment of youth’s strengths, there are limitations that must be considered. First, a selection bias may have been introduced based upon the inclusion criteria and purpose of this study. That is, there were minor, but statistically significant differences between the participants who were included in the current study and those who were excluded, based upon the inclusion criteria that required complete data on all three measures used for the current study (i.e., BERS-2, CBCL, CIS) and youth age between 6 and 18 years. Second, the current study included youth who were referred to community mental health settings due to emotional and/or behavioral concerns. Thus, findings may differ for youth being served in community health settings for other presenting problems. A final limitation is that the present study reports the findings from data that were collected from caregivers; therefore, it is possible that results may differ for youth or clinician ratings.

Future Directions

The promising results of the present study warrant continued research on the BERS-2 within mental health populations. One direction for future research is to better understand potential differences in the psychometric properties of the BERS-2 scores based upon youth age, diagnosis, or membership in certain groups (e.g., race/ethnicity, gender). Further, research should also examine the degree of congruence between ratings from multiple respondents, such as caregivers and youth. Moreover, it is likely that the BERS-2 is also administered at regular intervals over time, in order to monitor youths’ response to treatment. Therefore, future research should examine the development and stability of change in youths’ strengths over time as youth receive treatment for mental health concerns. One final avenue of future research is to examine the BERS-2 strength profiles of youth who make clinically significant improvement in symptoms of psychopathology and functional impairment versus those who do not, in order to better inform treatment planning and delivery.