Preschool referrals for mental health issues are not uncommon, with externalizing/disruptive behaviors (e.g., attention-deficit/hyperactivity disorder [ADHD], oppositional defiant disorder [ODD], conduct disorder [CD]) frequently cited as reasons for these referrals (Bufferd et al. 2012). In a survey conducted with pre-kindergarten teachers in the United States (US), approximately 10% reported expelling at least one preschooler in the last year (Gillam 2005). Moreover, boys in the US are at increased risk (when compared to girls) of being suspended from preschool, often multiple times, for disruptive behaviors (United States Department of Education 2014). In Taiwan, research suggests younger children within a grade (irrespective of whether the child is a boy or a girl) are at higher risk of being diagnosed and prescribed medication (methylphenidate or atomoxetine) than older children in the same grade (Chen et al. 2016). Other research in Taiwan suggests that boys present more often for behavioral treatment than girls (Gau et al. 2010; Huang et al. 2003). Research using the Chinese Kiddie-Schedule for Affective Disorders and Schizophrenia-Epidemiological version (Chinese K-SADS-E) with children diagnosed with ADHD comorbid with ODD/CD and their non-ADHD controls (Liu et al. 2017) suggested that, consistent with previous research in US children and adolescents (Hoza et al. 2004; Rapport et al. 1999), Taiwanese children and adolescents with ADHD had worse academic performance and attitude toward schoolwork, weaker social interactions, and more school behavioral problems than non-ADHD controls. They also tended to under-report their school adjustment (i.e., reporting themselves to have a better learning attitude and fewer school behavioral problems) than their parents. Given the negative outcomes associated with behavioral difficulties in childhood, early detection and intervention for disruptive behavior problems are essential to prevent or reduce adverse school outcomes in Taiwanese children and adolescents.

While many instruments have been validated for assessing disruptive behavior problems in North America and Europe (Burkey et al. 2016; Canino et al. 2010), only a few instruments have been developed for use in Taiwan. The Child Behavior Checklist (CBCL; Achenbach and Rescorla 2000, 2001), which is a broadband measure that allows for a global assessment of child behavior problems has established norms and psychometric properties for Chinese populations (Achenbach and Rescorla 2007, 2010; Yen et al. 2014). Unfortunately, the CBCL and other broadband measures of child behaviors often assess diagnostic characteristics of behavior problems such as ADHD, ODD, and CD. For example, the Chinese K-SADS-E is another broadband measure used in making ADHD, ODD, and CD diagnoses (Liu et al. 2017). Although several narrowband measures, which focus on a specific behavioral problem domain, have been translated into Chinese (e.g., the Chinese Swanson, Nolan, and Pelham, version IV scale–parent and teacher forms [Chinese SNAP-IV-P and SNAP-IV-T], the short form of the Chinese Conners’ Parent and Teacher Rating Scales-revised [Chinese CPRS-R:S and CTRS-R:S]; Gau et al. 2006, 2008, 2009), a narrowband, behavioral rating scale that assesses ADHD, ODD, as well as CD symptoms has not been developed for use with Taiwanese children. This presents a conundrum, as behavioral parent training (BPT) programs are often cited as the gold standard and best practice for addressing disruptive behaviors in young children because of the robust treatment effects found across a range of ages and externalizing behaviors (e.g., Chambless and Ollendick 2001; Eyberg et al. 2008; Forgatch et al. 2004; Sanders 2012; Kaminski et al. 2008; Webster-Stratton and Reid 2004). Its effectiveness/efficacy with and without cultural adaptation has been demonstrated for children with externalizing behaviors in various cultural groups (e.g., Baumann et al. 2015; Homen et al. 2015; Leung et al. 2017; Matos et al. 2009; Sumargi et al. 2015), including Taiwanese families (Chen and Fortson 2015; Huang et al. 2003). It is difficult, however, to deliver an evidence-based behavioral parent training program without measures that effectively assess for disruptive behavior disorders and/or that can be used in repeated assessments.

Although the mode of delivery may differ for the various BPT programs, the programs often include frequent assessments to guide treatment. The Eyberg Child Behavior Inventory (ECBI; Eyberg and Pincus 1999) is a frequently used narrowband, parent-report behavioral rating scale that assesses child disruptive behavior (including ADHD, ODD, and CD symptoms) and is often used in the assessment of treatment effects. The ECBI consists of 36 items and was developed for assessment of children between the ages of 2 and 16 years. It contains two scales: the Intensity and the Problem Scales. The Intensity Scale evaluates the frequency of common child behavior problems with items rated 1 (never) to 7 (always), while the Problem Scale assesses whether the behavior is a problem (or not) for the parent with items answered in a yes/no format (Eyberg 1992; Robinson et al. 1980).

Research has found the ECBI to have good psychometric properties in US populations and in other countries/cultures in which it has been used. Research has been mixed, however, with respect to whether the ECBI is a unidimensional or multidimensional measure (e.g., Abrahamse et al. 2015; Axberg et al. 2008; Coffey et al. 2015; Gross et al. 2007; Ismaili 2015; Reedtz et al. 2008; Rhee and Rhee 2015; Weis, 2005). Comparisons of US- and non-US-based samples as a means of understanding similarities and differences across cultures, are described in the sections that follow.

For the US-based samples, the Intensity and Problem Scales have been found to have good test-retest reliability, with reliability estimates ranging from 0.75 to 0.86 for the Intensity Scale and from 0.75 to 0.88 for the Problem Scale (Eyberg and Pincus 1999; Funderburk, 2003; Robinson et al. 1980). Excellent internal consistency has been obtained for both the Intensity (alphas ranged from 0.90 to 0.98) and Problem (alphas ranged from 0.91 to 0.98) Scales in the various child and/or adolescent normative and clinical samples in which it has been used (Bagner and Eyberg 2007; Coffey et al. 2015; Eyberg and Robinson1983; Gross et al. 2007; Robinson et al. 1980). The test-retest reliability of the Intensity and Problem scales for youth in South Korea were 0.92 and 0.97 (Rhee and Rhee 2015), 0.85 and 0.83 in Puerto Rico (Cumba, 2002), 0.84 and 0.60 in the Netherlands (Abrahamse et al. 2015), and 0.88 and 0.67 in Sweden (Axberg et al. 2008). The internal consistency for Albanian, Chinese, Dutch, Korean, Norwegian, Puerto Rican, and Swedish samples ranged from 0.91 to 0.95 on the Intensity scale and 0.88 to 0.94 on the Problem scale, indicating high internal consistency (Abrahamse et al. 2015; Axberg et al. 2008; Chen and Fortson 2015; Cumba et al. 2002; Ismaili 2015; Leung et al. 2003a, b; Reedtz et al. 2008; Rhee and Rhee 2015).

For the US-based samples, the ECBI was initially developed and supported for use as a unidimensional measure (for 35 of the 36 items) of externalizing behaviors in US children (Robinson et al. 1980) and adolescents (Eyberg and Robinson 1983) seen in a pediatric clinic. During restandardization of the ECBI, Colvin (1999a) examined the underlying factor structure and found a one-factor model for 33 of the 36 items. Coffey et al. (2015) conducted a confirmatory factor analysis and found a better fit for a 36-item one-factor model (when compared with a 22-item model found in other research [e.g., Burns and Patterson 1991, 2000; Weis et al. 2005] and described below). Other research on the psychometrics of the ECBI has suggested that it is a multidimensional measure for clinical and community samples of youth in the US. Burns and Patterson’s (1991) initial factor analysis on the ECBI resulted in three dimensions: (a) oppositional behaviors, (b) ADHD-related behaviors, and (c) conduct problem behaviors. A reanalysis (Burns and Patterson 2000) wherein two random samples were created and an exploratory factor analysis with oblique rotation was conducted with the first sample resulted in four factors: (a) oppositional defiant behavior toward adults, (b) ADHD-related behaviors, (c) conduct problem behaviors, and (d) unnamed factor. After selecting the items from the three clinically meaningful factors, a tripartite model was supported with confirmatory factor analysis in the second sample: (a) oppositional defiant behavior toward adults, (b) inattentive behavior, and (c) conduct problem behaviors. Weis et al. (2005) confirmed the three-factor structure of the ECBI in a community sample of young children aged 2 to 6 years. In the studies of Burns and Patterson (1991; 2000) and Weis et al. (2005), an adequate (and more significant) fit for the three-factor model was observed for the ECBI when compared to the one-factor model.

For the non-US-based samples, confirmatory factor analyses supported results similar to those found in US populations, as a three-factor, 22-item model has been observed in a Swedish population (Axberg et al. 2008). In a Korean population, Rhee and Rhee (2015) found an eight-factor, 36-item model using an exploratory factor analysis: (a) ADHD behavior, (b) disruptive behavior toward adults, (c) disruptive behavior expressed overtly, (d) problems of daily living–clothing and eating, (e) disruptive behavior toward peers and expressed passively, (f) problems of daily living–sleeping, (g) problems in sibling relationships, and (h) stigmatized behavior. The measure was also observed to have a unidimensional (i.e., 36-item) structure in a Dutch community and clinical sample (Abrahamse et al. 2015).

For US samples, research has established adequate convergent validity of the Intensity (Boggs, 1990; Coffey et al. 2015; Gross et al. 2007) and Problem scales (Boggs, 1990; Gross et al. 2007) with the CBCL Externalizing scale (Achenbach and Edelbrock 1983; Achenbach and Rescorla 2000, 2001; rs ranged from 0.54 to 0.73), the parent version of the Preschool Behavior Questionnaire (Behar and Stringfield 1974; Funderburk et al. 2003; rs ranged from 0.34 to 0.53), and on the Parenting Stress Index (PSI; Abidin 1995; Eyberg, 1992; rs ranged from 0.59 to 0.62). The strong, significant correlations of the ECBI and other measures of child disruptive behavior and the weaker correlations of the ECBI and measures of internalizing behavior are suggestive of the divergent validity of the measures. For example, in a study of US children and adolescents ages 4 to 16 referred for psychological evaluation, the correlations between the ECBI Intensity and Problem scales with the CBCL Internalizing scale (r = 0.41 and r = 0.48, respectively) were lower than those for the CBCL Externalizing scale (r = 0.73 and r = 0.65, respectively), indicating the divergent validity of the ECBI and the CBCL Internalizing scale (Boggs et al. 1990).

For non-US-based samples, convergent validity was found in Korean and Taiwanese samples for the ECBI Intensity and Problem scales and the CBCL Externalizing scale (Achenbach and Rescorla 2000, 2001; Chen and Fortson 2015; Rhee and Rhee 2015; rs ranged from 0.34 to 0.85). The convergent validity of the ECBI Intensity (rs = 0.48 to 0.75) and Problem Scales (rs = 0.36 to 0.62) and subscales measuring conduct problems, hyperactivity, and impulsiveness on the Strengths and Difficulties Questionnaire (SDQ; Goodman 1997) was established for a Dutch clinical sample (Abrahamse et al. 2015). In terms of the divergent validity of the ECBI in a sample of Korean children, the correlations for the ECBI Intensity and Problem scales and the CBCL Internalizing scale (r = 0.50 and r = 0.35, respectively) were weaker than for the ECBI and the CBCL Externalizing scale (r = 0.85 and r = 0.58, respectively; Rhee and Rhee 2015). Weak correlations also were observed for the ECBI Intensity and Problem scales and subscales of the SDQ not measuring externalizing behaviors (e.g., Emotional Symptoms, rs = 0.12 to 0.37; Peer Problems, rs = 0.03 to 0.14; and Prosocial Behavior, rs = −0.10 to −0.44; Abrahamse et al. 2015).

For the US-based samples, multiple studies also have shown that the ECBI can discriminate between children with and without clinically elevated levels of externalizing behaviors (e.g., Aragona and Eyberg 1981; Baden and Howe 1992; Eyberg and Ross 1978; McNeil et al. 1991; Rich and Eyberg 2001). For example, the ECBI has been found to discriminate (a) between clinic-referred children and non-problem children, including clinic-referred children who did and did not receive treatment of behavior problems (Colvin et al. 1999a) and (b) between medically ill children with and without histories of treatment or referral for learning disabilities and/or behavior problems (Colvin, 1999b). For the non-US-based samples, Rhee and Rhee (2015) established the discriminative validity of the ECBI Intensity and Problem scales for Korean children who did/did not receive mental health treatment at community counseling centers. The ECBI Intensity scale also was found to discriminate between children who did and did not meet diagnostic criteria for a disruptive behavior disorder in the Netherlands (Abrahamse et al. 2015).

The predictive validity of the ECBI has received limited attention. In a US-based study, Rich and Eyberg (2001) examined the predictive power of the ECBI Intensity scale using a cut-off score of 132 to compare 98 American children who met diagnostic criteria for a disruptive behavior disorder and 98 non-diagnosed American children. Rich and Eyberg found the overall correct classification rate of the ECBI to be 0.91, with a sensitivity of 0.96, a specificity of 0.87, a positive predictive power of 0.88, and a negative predictive power of 0.96.

Although Chen and Fortson (2015) reported excellent internal consistency for the ECBI Intensity and Problem scales (Cronbach’s alphas = 0.94 and 0.91, respectively) and adequate convergent validity between the ECBI Intensity and Problem scales (r = 0.68 and r = 0.34, respectively) and the CBCL Externalizing Problem scale for a sample of Taiwanese children, the factor structure and discriminative validity of the ECBI has not been examined with Taiwanese children. Due to the necessity of improving child mental health practice in Taiwan, it is important to examine validity of the ECBI as a concise measure of child disruptive behavior. The purpose of this study was to examine the factor structure and to establish the discriminative validity of the ECBI with a sample of Taiwanese clinic-referred children with clinically elevated externalizing behavior problems and a matched comparison group. The current study also sought to examine the convergent and divergent validity of the ECBI with the CBCL in each of the samples.

Method

Participants

Participants of this study included (a) 70 Taiwanese caregivers aged 23 to 66 years referred for treatment of clinically significant externalizing behavior problems in their 2- to-11-year children and (b) 70 demographically matched comparison caregivers. The matched caregivers were a non-clinic referred sample of 29- to 68-year-old Taiwanese caregivers with children between 2- and 11-years-old. The two samples were pairwise matched on the child’s age and gender, caregiver’s gender, and region of Taiwan where they resided (northern vs. southern Taiwan). Caregivers of all children seen in the clinic signed an informed consent form allowing data from their records to be used for research purposes. The demographic characteristics of each sample are presented in Table 1.

Table 1 Demographic characteristics of clinic-referred and matched comparison samples in Taiwan
Full size table

Procedure

The 2- to 11-year-old clinic-referred sample was recruited from northern and southern Taiwan. Children who had disruptive behavior problems that led the parents to seek treatment in outpatient clinics in local hospitals were referred by mental health professionals, including child psychiatrists, clinical psychologists, and social workers. A total of 100 clinic-referred caregiver-child dyads completed a research packet containing an informed consent form, a demographic questionnaire, and measures of child and parent functioning. Of the 100 clinic-referred caregiver-child dyads, 70 were included in the present study after being pairwise matched to the comparison sample based on the children’s age, gender, and place of residence.

The demographically matched sample consisted of 70 caregivers recruited from the same communities as the clinic-referred sample in northern and southern Taiwan through convenience sampling. The interested caregivers were self-referred in response to (a) flyers to caregivers in public areas (e.g., indoor or outdoor playgrounds, bookstores, front entrances of preschools/kindergartens and elementary schools, churches), (b) word of mouth, and/or (c) internet message boards. The interested caregivers were mailed and completed the same research packet as the caregivers of the clinic-referred sample. To be included in the study, children had to be of normal intelligence and had not been referred for treatment of disruptive behavior problems. Every clinic-referred and comparison caregiver-child dyad received approximately $300 New Taiwan dollars (NTD), which is equal to approximately $10 USD, when the caregiver’s questionnaire packet was completed and returned. The ECBI (Eyberg and Pincus 1999) and CBCL (Achenbach and Rescorla 2007, 2010) were the only caregiver report measures used for the current study. All ECBI and CBCL measures were completed with no item omissions; thus, there was no missing data.

Measures

A parental demographic questionnaire was designed to assess child, caregiver, and family characteristics (e.g., caregiver’s and child’s age, gender, ethnicity, educational status of the caregivers, family status and income) and other demographics (e.g., medical and mental health history of the caregiver and the child). In addition to the ECBI, the CBCL for ages 1.5–5 (Achenbach and Rescorla 2000) and the CBCL for ages 6–18 (Achenbach and Rescorla 2001) were used to assess for emotional and behavioral problems in youth. For purposes of the current research, only the Syndrome Scale, which yields a Total Problem score and two broadband scores (Internalizing and Externalizing Problem scores), was used. Items are rated on a 3-point scale (0 = not true, 1 = sometimes true, and 2 = very true or often true; Achenbach and Rescorla 2000, 2001). Multicultural norms, which includes normative samples from Taiwan and other societies, and cutoff scores have been established for the measure (Achenbach and Rescorla 2007, 2010; Chen et al. 2009). The internal consistency (αs ranging from 0.55 to 0.90 for the CBCL 6–18; αs ranging from 0.62 to 0.95 for the CBCL 1.5–5), test-retest reliability (rs ranging from 0.51 to 0.74 for the CBCL 6–18 one-month interval; rs ranging from 0.52 to 0.84 for the CBCL 1.5–5), interparental agreement (0.25 to 0.84 for the CBCL 1.5–5), and construct validity (eight-factor structure for the CBCL 6–18; seven-factor structure for the CBCL 1.5–5) have been demonstrated (Yang, 2000; Wu et al. 2012).

For this study, permission was granted to the first author by Psychological Assessment Resources, Inc (PAR), USA to translate the ECBI into Traditional Chinese for Taiwan to examine its psychometric properties. PAR approved forward and back-translations of the ECBI.

Data Analysis

Demographic data

Descriptive statistics (e.g., means and frequencies) were calculated to describe the demographics and the variables assessing the caregiver’s and the child’s functioning (see Table 1) at the time of assessment. Chi-square tests and independent two-sample t-tests were conducted to examine group differences (clinic-referred vs. matched control samples) on demographic variables and the variables assessing the caregiver’s and child’s functioning at the time of assessment. Because 10 independent two-sample t-tests were performed simultaneously, a Bonferroni-adjusted significance level of 0.005 was used to reduce the possibility of Type I error (Bland and Altman 1995).

Factor structure

Because the dimensional structure of the ECBI is unclear (e.g., Burns and Patterson 1991, 2000; Eyberg and Robinson 1983; Robinson et al. 1980; Rhee and Rhee 2015; Weis et al. 2005), particularly as applied in a Taiwanese population, exploratory factor analyses (using maximum likelihood extraction) were performed on the ECBI Intensity items. Because of the high levels of comorbidity often observed for the various disruptive behavior disorders (Mash and Wolfe 2016), which past research has suggested comprise the multidimensional factor structure of the ECBI, a varimax rotation was used for correlations ranging from −0.3 to 0.3; otherwise, an oblique rotation was used and maximum likelihood factors extracted.

Convergent and divergent validity

Pearson’s correlation coefficients were conducted to examine the convergent validity of the ECBI Intensity and Problem scale scores and the CBCL Externalizing Problem scale scores for each sample. Pearson’s correlation coefficients also were conducted to examine the divergent validity of the ECBI Intensity and Problem scale scores and the CBCL Internalizing Problem scale scores for each sample.

Discriminative validity

Independent two-sample t-tests were conducted to examine whether the ECBI Intensity and Problem scale scores were able to differentiate 45 clinic-referred children with clinically elevated externalizing problems from 45 children in the matched comparison sample.

Results

Demographic Data

No significant group differences for the clinic-referred vs. the matched comparison samples were found for any of the child, caregiver, and family demographic characteristics (e.g., child’s age and gender, caregiver’s gender, and location of residence; see Table 1). Compared to children and caregivers in the matched comparison sample, children (50 vs. 1.4%), χ2(1, N = 140) = 132.22, p < 0.001, and caregivers (12.9 vs. 0%), χ2(1, N = 140) = 20.66, p < 0.001, in the clinic-referred sample were more likely to receive mental health services for psychological problems/disorders.

At the time of assessment, the caregivers of the clinic-referred sample (as compared to the matched comparison sample) reported their children as having significantly higher levels of emotional and behavioral difficulties, which was expected based on how families were recruited. For example, the clinic-referred sample rated their children as having higher levels of disruptive behaviors on the ECBI, t(138) = 16.67, p < 0.001, rated their children as having more disruptive behaviors that were a problem for them on the ECBI, t(138) = 15.29, p < 0.001, and rated their child as having more overall emotional and behavioral problems on the CBCL Total Problems scale, t(138) = 13.94, p < 0.001, more externalizing problems on the CBCL Externalizing Problems scale, t(138) = 13.97, p < 0.001, and more internalizing problems on the CBCL Internalizing Problems scale, t(138) = 10.25, p < 0.001.

Factor Structure

According to MacCallum, (2001), when the communalities of variables are high, good factors can be obtained with a relatively small sample, as it suggests that variance in the variables has been extracted by the factor solution. In the current study, Kaiser-Meyer-Olkin (KMO) tests and initial communality values of the ECBI items were used to examine the sampling adequacy and to determine whether the small sample sizes included in this study were likely to distort the factor solutions (Kaiser and Rice 1974; Norusis 2010). In general, KMO values less than 0.50 indicate unacceptable sampling adequacy, values greater than 0.50 but less than 0.60 indicate miserable sampling adequacy, and values greater than 0.60 but less than 0.70 indicate mediocre sampling adequacy (Kaiser and Rice 1974; Norusis 2010). We found KMO values to be 0.53 for the clinic-referred sample and 0.69 for the matched comparison sample. Although these values do not eliminate all doubt as to the sample adequacy, they are marginally acceptable (Kaiser and Rice 1974; Norusis 2010). When each of the ECBI items was examined individually across the clinic-referred and matched comparison samples, only 1 item (item 33) in the clinic-referred sample, which had a value of 0.47, had an initial communality value below the 0.50 threshold. Thus, the small sample size is not likely to impact the factor solutions (MacCallum et al. 2001; Norusis 2010).

For the clinic-referred sample, the results of an exploratory factor analysis showed that a six-factor model using maximum likelihood extraction with varimax rotation provided the most clinically useful dimensions. The six eigenvalues greater than one accounted for a total of 49.97% of the variance on the 36 ECBI Intensity scale items (see Table 2). Moreover, the correlations among the six factors were low, ranging from –0.23 to 0.29 (only three correlations > .20). The six factors included the following: (1) self-discipline (problems in daily activities including clothing, eating, sleeping), (2) non-destructive or non-aggressive defiance to parents or adults, (3) conduct problems—destructive or aggressive behaviors toward objects or people, (4) conduct problems—violation of social regulations, (5) inattentive behaviors, and (6) sibling teasing or fighting. Items with factor loadings equal to or greater than .33, which are considered part of one factor and not others, are shown in boldface in Table 2.

Table 2 Factor Loadings for Exploratory Factor Analysis with Maximum Likelihood Extraction and Varimax Rotation of the ECBI Intensity Scale for the Clinic-referred Sample
Full size table

For the matched comparison sample, the results of an exploratory factor analysis showed that a five-factor model using maximum likelihood extraction with an oblique rotation provided the most clinically useful dimensions, accounting for a total of 50.45% of the variance. Moreover, the correlations among the five factors were low to moderate, ranging from 0.32 to 0.52. The five factors included the following: (1) self-discipline (including problems in daily activities and minor violation of social regulations), (2) non-destructive or non-aggressive defiance to parents or adults, (3) ADHD-related behaviors and conduct problems, (4) sibling teasing or fighting, and (5) sleeping problems. Items with factor loadings equal to or greater than 0.33, which are considered part of one factor and not others, are shown in boldface in Table 3. Because the two samples had different factor structures and different items loaded on each factor, the samples were not combined to examine the overall factor structure or item loadings. For both samples, three items had factor loadings of less than .33 and did not fit into any of the factors: has difficulty entertaining self alone, cries easily, and wets the bed.

Table 3 Factor Loadings for Exploratory Confirmatory Factor Analysis with Maximum Likelihood Extraction and Oblique Rotation of the ECBI Intensity Scale for the Matched Comparison Sample
Full size table

Convergent and Divergent Validity

Adequate convergent validity was found between the ECBI Intensity scale and the CBCL Externalizing Problem scale for the clinic-referred sample, r = 0.48, p < 0.01, and the matched comparison sample, r = 0.57, p < 0.01. The ECBI Problem scale also showed adequate convergent validity with the CBCL Externalizing Problem scale for the matched comparison sample, r = 0.41, p < 0.01. The correlations between the ECBI Problem scale and the CBCL Externalizing Problem scale were not significant for the clinic-referred sample, r = 0.10, p = 0.40. To examine the divergent validity of the ECBI with other measures of emotional distress, the ECBI Intensity and Problem scales were compared to the CBCL Internalizing Problem scale. A weak but significant correlation was observed between the ECBI Intensity Scale and the CBCL Internalizing Problem scale for the matched comparison sample, r = 0.43, p < 0.01. The correlation between the ECBI Intensity scale and the CBCL Internalizing Problem scale for the clinic-referred sample was not significant, r = 0.22, p = 0.07. The correlations between the ECBI Problem scale and the CBCL Internalizing Problem scale were not significant for the clinic-referred sample, r = 0.18, p = 0.13, or for the matched comparison sample, r = 0.23, p = 0.06.

Discriminative Validity

The discriminative validity of the ECBI Intensity and Problem scales was examined with the clinic-referred children (n = 45) and their matched comparisons (n = 45). The 45 clinic-referred children with clinically elevated externalizing problems had significantly higher scores on the ECBI Intensity scale (M = 154.27, SD = 20.73) as compared to the matched comparison sample (M = 82.36, SD = 18.39), t(88) = 17.41, p < 0.001. They also had significantly higher scores on the ECBI Problem scale (M = 22, SD = 5.32) as compared to the matched comparison sample (M = 4.11, SD = 4.10), t(88) = 17.87, p < .001). In general, the ECBI Intensity and Problem scales were able to differentiate the 45 clinic-referred children with clinically elevated externalizing problems from the matched comparison sample who had no history of referrals for behavior problems.

Discussion

Although several narrowband behavioral rating scales (i.e., Chinese CPRS-R:S and CTRS-R:S, Chinese SNAP-IV-P and SNAP-IV-T) and a broadband behavioral rating scale (i.e., Chinese CBCL) have been translated into Chinese and have established norms and psychometric properties (Achenbach and Rescorla 2007, 2010; Gau et al. 2009, 2008, Gau et al. 2008; Gau et al. 2006; Yen et al. 2014), they either only address ADHD and ODD symptoms or consist of items not specific enough for assessing disruptive behaviors. The need for narrowband and psychometrically sound behavioral rating scales for ADHD, ODD, as well as CD symptoms in Taiwanese children is great. The current study sought to examine the validity (i.e., factor structure, convergent, divergent, and discriminative validity) of the ECBI (a narrowband behavioral rating scale) in a sample of treatment-seeking caregivers and their children and a matched comparison group of caregivers and their children recruited from the community. No differences were observed in the demographics between samples, suggesting that the two samples were appropriately matched on those variables. As would be expected given the nature of referral to the two samples, caregivers of children in the clinic-referred sample reported higher levels of externalizing/disruptive behaviors, emotional and behavioral problems overall, and internalizing behaviors.

In the examination of the factor structure of the 36-item ECBI, the exploratory factor analyses in the present study resulted in a six-factor model for the clinic-referred sample and a five-factor model for the matched comparison sample. These results differ from previous studies which found various factor structures of the ECBI, including a one-factor structure using the 36-item ECBI in US (Colvin et al. 1999a; Coffey et al. 2015; Eyberg and Robinson 1983; Robinson et al. 1980) and Dutch (Abrahamse et al. 2015) samples, a three-factor structure using 22 (Axberg et al. 2008; Burns and Patterson 2000; Weis et al. 2005) and 36 (Burns and Patterson 1991) items in US and Swedish populations, and an eight-factor structure using 36 items in a Korean sample (Rhee and Rhee 2015). The mixed results observed across the various studies conducted in the same or different countries/societies suggest that the factor structure of the ECBI may be dependent on the characteristics (e.g., differences in psychopathology, ethnicities, nationalities, and cultures) of the target populations being assessed. Because the initial communality values for some ECBI items were marginally acceptable for both samples in the current study (see Tables 2 and 3), the factor structures of the ECBI should be confirmed with larger clinical and non-clinical samples in Taiwan. Future research also is needed to determine whether the factor structure of the ECBI remains consistent across subgroups in the same country/society or different countries/societies and whether countries/societies (e.g., Taiwan and South Korea) sharing similar cultures (e.g., a Confucian philosophical worldview) may have a more similar factor structure for the ECBI than other countries who do not.

As in previous research (e.g., Abrahamse et al. 2015; Burns and Patterson 1991), one item (i.e., wets the bed) did not load on any of the factors for the matched comparison sample. For the clinic-referred sample, the item wets the bed had a low loading (.32) on factor 6, Siblings teasing or fighting (see Table 2), but this loading did not have a clinically meaningful fit. An additional item (i.e., has difficulty entertaining self alone) in the current study did not load on any of the factors for the clinic-referred and matched comparison samples, one item (i.e., cries easily) did not load on any factors for the clinic-referred sample, and a final item (i.e., destroys toys and other objects) did not load on any of the factors for the matched comparison sample. These results suggest that variations exist in how caregivers perceive the behaviors of children, even among clinic and non-clinic referred families in the same cultural group. The results provide important information for clinical practice about the types of behaviors that may/may not be considered disruptive by the caregivers of Taiwanese clinic-referred and/or non-clinic-referred children.

When taking a closer look at the factor loadings, several items for the clinic-referred sample are loosely associated with their loaded factors. For example, item 35, is overactive or restless, loaded on factor 3, conduct problems—destructive or aggressive behaviors toward objects or people, rather than where one might have expected, such as factor called ADHD/ADHD behavior (Burns and Patterson 1991, 2000; Rhee and Rhee 2015) or on no factor (Axberg et al. 2008; Weis et al. 2005), as was found in previous research. It is possible that the item loaded on the conduct problems factor in the current research because it is hard for a child with ADHD to control their behavior and/or pay attention. When a child is overactive or restless, he/she may have a tendency to engage in play fight, horseplay, or other rough play that leads to destructive or aggressive behaviors toward objects or people. Moreover, item 28, constantly seeks attention, loaded on factor 4, conduct problems – violation of social regulations, rather than on factors identified in previous research where one might have expected such items to load (e.g., ADHD/ADHD behavior [Burns and Patterson 1991], disruptive behavior toward peers and expressed passively (Rhee and Rhee 2015), no factor/unnamed factor (Axberg et al. 2008; Burns and Patterson 2000; Weis et al. 2005). The reason for this is likely cultural, as constantly seeking attention, such as by interrupting people’s conversations (e.g., to ask for adults to watch them do tricks, dance, play), is considered inappropriate and impolite in Chinese culture (Tamis-LeMonda, 2002). Parents may feel embarrassed (described by Chinese as “losing face”) by their children’s behavior, which may be considered a violation of social regulations in the culture.

When we examined the factor structure of the ECBI for the clinic-referred and matched comparison samples in the present study, similar factors were identified: (a) self-discipline, (b) non-destructive or non-aggressive defiance to parents, and (c) adults and sibling teasing or fighting. The factor structure for the clinic-referred sample had two factors related to conduct problems (i.e., destructive or aggressive behavior toward objects or people and violation of social regulations) that distinguish it from the matched comparison sample. All items in the ADHD-related behaviors and conduct problems factor and most of the items in the self-discipline factor for the matched comparison sample were divided into three different factors in the clinic-referred sample. This suggests that the factor structure of the ECBI for the clinic-referred sample may be more dimensional than the matched comparison sample, which helps in understanding and perhaps differentiating the variety of clinically elevated disruptive behaviors that may be reported by caregivers. As the ECBI is used for Taiwanese clinic-referred children, the six subscales may result in the ECBI being more sensitive to assess intervention effects on different types of disruptive behavior problems, particularly on conduct problems.

As with previous research conducted in US populations (Boggs et al. 1990; Gross et al. 2007; Coffey et al. 2015) and in other countries/cultures (Chen and Fortson 2015; Rhee and Rhee 2015), adequate convergent validity was observed for the ECBI Intensity and the CBCL Externalizing Problem scales for the clinic-referred and matched comparison samples. The ECBI Problem scale also had adequate convergent validity with the CBCL Externalizing Problem scale for the matched comparison sample. These results suggest that the ECBI, like the CBCL, which has had extensive psychometric validation and normative evaluations across various populations (Achenbach and Rescorla 2000, 2001, 2007, 2010), is a valid measure for assessing child behavioral problems. Given that convergent validity was not established for the ECBI Problem scale and the CBCL Externalizing Problem scale in the clinical sample, the ECBI Intensity scale scores should be used primarily in the assessment of disruptive behavior problems in Taiwanese clinical samples, and future research should continue to examine the psychometrics of the scale (particularly the ECBI Problem scale) in clinical populations.

Good discriminative validity of the ECBI Intensity and Problem scales in Taiwanese children was established, as the measure clearly differentiated those with clinically significant levels of behavior problems and those with minimal or “typical” issues. The established discriminative validity of the ECBI replicated the findings of previous research conducted with US populations (Aragona and Eyberg 1981; Baden and Howe 1992; Eyberg and Ross 1978; McNeil et al. 1991; Rich and Eyberg 2001) and in other countries/cultures (Abrahamse et al. 2015; Rhee and Rhee 2015). Because the ECBI is a caregiver-report measure, it is likely influenced by the caregiver’s subjective responses toward the child so caution must be exercised when no other data sources or collection methods are available to confirm the assessment results of this measure.

Limitations and Future Research

The validation of the ECBI in the current study may have been limited by a number of factors, including the small sample size, the different recruitment methodologies for the two groups (i.e., convenience sampling in the matched comparison sample and recruitment for treatment in the clinic-referred sample), and the fact that recruitment occurred in different locations in Taiwan (both northern and southern Taiwan), although the latter recruitment methodology was consistent across the two groups. The small sample size in the current study limits the ability to conduct additional analyses, such as two-way (contingency) table analyses to further examine discriminative validity. The small sample size also limits the ability to use ECBI component scores, based on the results of factor analyses, to differentiate within the spectrum of externalizing behavior problems due to the small numbers of participants who would fall into a single behavioral disorder category. Another limitation is that the present study did not provide test-retest data, which has been established in several countries/cultures including the US (e.g., Funderburk et al. 2003), South Korea (Rhee and Rhee 2015), Puerto Rico (Cumba et al. 2002), the Netherlands (Abrahamse et al. 2015), and Sweden (Axberg et al. 2008). Finally, the clinic-referred and matched comparison samples in the present study were selected based on the use of one data collection method (i.e., other-report measure) and one source (the child’s caregiver). Use of multiple data collection methods and sources (Kazdin 2003) would be helpful in future examinations of the psychometrics of the ECBI.

In general, these results suggest that the ECBI is a valid measure of behavior problems in Taiwanese children. The measure is not unidimensional, as the dimensions appeared to change depending on the sample being utilized. Good convergent and divergent validity was established for the measure. The ECBI Intensity scale (when compared to the ECBI Problem scale) has better convergent validity with the CBCL Externalizing scale in assessing disruptive behaviors for the clinic-referred and matched comparison samples and divergent validity with the CBCL Internalizing scale. Both the ECBI Intensity and Problem scales, which address the frequency of child behavior problems and parental distress caused by child behavior problems, are valid to discriminate Taiwanese children with and without clinically elevated levels of externalizing behaviors. Future research may seek to replicate, refine, and/or further examine the findings of the present study with larger clinical and non-clinical samples and different recruitment/sampling methods that incorporate multiple informants and other regions in Taiwan. Future studies are also needed to examine other psychometrics (e.g., test-retest reliability) of the ECBI and to establish the normative data of the ECBI in Taiwan. In addition, as noted earlier, BPTs have been adapted and are being used in other countries, including Taiwan. Although additional research on the ECBI is needed, this study suggests that the ECBI is appropriate for measuring behavior problems in Taiwanese children and may be used in the context of BPTs to assess treatment progress. The ECBI also may serve as a valuable screening tool to use in early identification of disruptive behaviors in young Taiwanese children.

Disclaimer

The findings and conclusions in this manuscript are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention.