Disruptive behavior disorders (DBDs) are relatively common childhood mental health concerns, with a worldwide prevalence of approximately 6% (Polanczyk et al. 2015) and an estimated lifetime prevalence of between 5 and 15% in childhood (Merikangas et al. 2010). DBDs include oppositional defiant disorder (ODD) and conduct disorder (CD) and are often comorbid with attention-deficit/hyperactivity disorder (ADHD). Children with DBDs are a heterogeneous population who present with varying levels of symptom severity and impairment (Aitken et al. 2018; Musser et al. 2013; Waschbusch et al. 2004; Waschbusch and Willoughby 2008a). Rates of comorbid psychopathology are also high among children with DBD, including heterotypic comorbidity with internalizing disorders such as anxiety and depression (Greene et al. 2002). The present study examined the extent to which comorbid internalizing symptoms are associated with differences in the severity of children’s DBD, with the overall goal of informing both clinical services and theoretical models of DBD.

Internalizing Symptoms and Disruptive Behavior Disorders

Between 30 and 50% of clinic-referred children with DBD meet criteria for major depressive disorder, and approximately 40% meet criteria for multiple anxiety disorders (Greene et al. 2002). For some children, internalizing problems precede the development of disruptive behavior, whereas for others, disruptive behavior emerges first, after which internalizing problems develop (Drabick et al. 2010). A dual pathway model has been proposed, in which comorbid internalizing symptoms may be associated with more severe disruptive behavior or less severe disruptive behavior (Drabick et al. 2010). These opposing hypotheses are referred to, respectively, as the multiple problems hypothesis and the buffer hypothesis (Drabick et al. 2010).

According to the multiple problems hypothesis, children with DBD who have higher levels of internalizing symptoms may have increased amygdala responses to threat and poorer limbic system regulation, thus contributing to poorer self-regulation and response selection (Drabick et al. 2010). Consistent with the multiple problems hypothesis, some studies have found that children with high levels of both disruptive behavior and internalizing symptoms are more impaired (Cunningham et al. 2013) and more likely to engage in delinquent behavior over time, than are children with disruptive behavior only (Garai et al. 2009). Similarly, children with ADHD and comorbid internalizing disorders show higher levels of disruptive behavior than do children with ADHD only (Humphreys et al. 2012) and are at risk for greater psychiatric morbidity over time, including suicide attempts (Biederman et al. 1991).

In contrast, the buffer hypothesis suggests that experiencing comorbid internalizing symptoms may be associated with less severe disruptive behavior (Drabick et al. 2010). According to the buffer hypothesis, children with higher levels of internalizing symptoms may engage in less disruptive behavior because they are anxious about the possibility of negative evaluations, or because they experience lower levels of behavioral activation (Cunningham et al. 2013; Garai et al. 2009). Consistent with the buffer hypothesis, Walker et al. (1991) found that clinic-referred boys with CD and comorbid anxiety had less problematic peer sociometric ratings, less contact with police, and fewer school suspensions than boys with CD only. Similarly, in some studies, higher levels of social anxiety in early adolescence have been associated with lower levels of conduct problems in early adulthood, controlling for baseline levels of conduct problems (Pine et al. 2000). While the multiple problems hypothesis appears to have more consistent empirical support, evidence remains mixed regarding whether comorbid internalizing symptoms exacerbate or mitigate the effects of DBDs concurrently and over time.

The buffer hypothesis was developed primarily regarding comorbid disruptive behavior and anxiety; however, in school-age children, the overlap in anxiety and depressive symptoms is so large that they are often statistically indistinguishable (Cole et al. 1997). As a result, childhood anxiety and depression are often referred to collectively as internalizing problems (Achenbach and Edelbrock 1978). Indeed, some investigations of the buffer and multiple problems hypothesis use a combined anxiety and depression score (e.g., Garai et al. 2009), an approach we have adopted here that is in keeping with empirically-derived models of child psychopathology (Achenbach and Edelbrock 1978; Martel et al. 2016).

The presence of CU traits, including low levels of guilt, remorse, concern for others, or concern about performance in important activities (Frick et al. 2005; Frick et al. 2014), is associated with a more severe and persistent course of disruptive behavior (Frick et al. 2014). However, there appears to be an inverse relationship between CU traits and internalizing comorbidity, with higher levels of internalizing, being associated with lower levels of CU traits (Frick and White 2008; Hawes and Dadds 2005). For example, Pardini and Fite (Pardini and Fite 2010) found that CU traits were associated with decreases in internalizing symptoms over time.

Gender Differences and Disruptive Behavior

Gender differences in rates of disruptive behavior have been consistently reported, with boys being two to three times more likely to meet criteria for DBDs than girls (Copeland et al. 2011; Martel 2013; Maughan et al. 2004; Wolraich et al. 2014). However, when girls do meet criteria for DBD, they may be more likely than boys to exhibit comorbid internalizing symptoms (Maughan et al. 2004; Merikangas et al. 2010; Zahn-Waxler et al. 2008). The higher rates of heterotypic comorbidity in girls are consistent with a “gender paradox” hypothesis, in which girls are less likely to have a DBD but exhibit more severe forms of the disorder and higher rates of comorbidity when they do (Keenan et al. 1999; Loeber et al. 2000). Thus, gender is an important consideration that remains under-explored in tests of the dual pathway model of anxiety and DBDs.

The Current Study

This study examined whether clinic-referred children with DBD with and without clinically elevated internalizing symptoms differ in terms of the severity of their disruptive behavior. Possible gender effects were also examined. In addition to behavioral ratings of disruptive behavior, two other indicators of DBD severity were used, which may be differentially related to comorbid internalizing symptoms: (1) impairment; and (2) callous-unemotional (CU) traits. Impairment ratings were included in order to provide additional information on DBD severity, given that behavioral symptoms and impairment provide related but unique information (Gadow et al. 2013; Rapee et al. 2012).

We expected that, consistent with the multiple problems hypothesis, children with DBD with clinically elevated internalizing symptoms would have higher levels of oppositional defiant behaviors and overall behavioral impairment. However, we expected that elevated internalizing symptoms would be associated with lower levels of CU traits in our sample. Analyses including gender were exploratory and therefore no specific hypotheses were made.

Method

Participants

Participants were 284 children ages 6 to 12 years (M = 9.2, SD = 1.6; 33 6-year-olds, 41 7-year-olds, 49 8-year-olds, 70 9-year-olds, 46 10-year-olds, 32 11-year-olds, and 13 12-year-olds; 76% male) and their caregivers, who were referred by a physician because of challenging child behavior to a specialized clinic for children with disruptive behavior in an urban mental health center in Canada and who had data available on the measures of interest. Respondent parents were mothers (86%), fathers (13%) or grandmothers (1%). The current study uses data from pre-treatment intake sessions. Caregivers’ self-reported primary ethnicity and education was as follows: 59% European origins; 18% North American origins; 8% Caribbean origins; 6% Latin, Central, and South American origins; 2% East and Southeast Asian origins; 2% Aboriginal origins; 2% African origins; and Other, 2%. The majority of caregivers (91%) reported a high level of confidence with reading and writing in English, with remaining caregivers reporting a medium level of confidence (7%) and a small number (2%) reporting a low level of confidence. The majority of the caregivers reported having completed post-secondary education (69%), with remaining caregivers reporting having completed up to the Grade 8 (1%), Grade 9 to Grade 12 (12%), or some postsecondary education (18%). Prior to data collection, caregivers provided informed consent and children provided assent. Paper and pencil questionnaires were given to the caregivers to complete in the presence of trained research staff. Measures used in the present study were part of a larger pre-treatment battery administered individually or in groups. All procedures in this study that involved human participants were approved by the institutional research ethics board and are in accordance with the 1964 Helsinki declaration and its later amendments.

Measures

Internalizing symptoms

Parent reports on the Emotional Symptoms scale of the Strengths and Difficulties Questionnaire (SDQ; Goodman 1997) were used to measure internalizing symptoms. The Emotional Symptoms scale consists of 5 items measuring anxiety, sadness, and physical symptoms, each rated as 0 (not true), 1 (somewhat true), or 2 (certainly true). The scale has demonstrated adequate internal consistency (α = 66; composite reliability = 0.80), test-retest reliability (r = 0.66), and correlations with other measures of anxiety and depression (r = 0.67 with the Children’s Depression Inventory; r = 0.73 with the total score on the Revised Children’s Manifest Anxiety Scale (Aitken et al. 2015; Muris et al. 2003; Stone et al. 2010) in previous studies. In the present sample, reliability for the Emotional Symptoms scale was acceptable, α = 0.71. Based on the SDQ scoring guidelines (www.sdqinfo.org), scores equal to or below 3 are considered normal, scores of 4 are considered borderline, and 5 or above are abnormal. Children were classified as exhibiting clinical levels of internalizing symptoms if their scores fell within the abnormal range on the Emotional Symptoms scale.

Disruptive behavior severity

Mean parent ratings on the Oppositional-Defiant subscale of the modified IOWA Conners (Loney and Milich 1982; Waschbusch et al. 2004) were used to measure the severity of children’s disruptive behavior. The IOWA Conners is a brief and efficient behavioral rating scale containing items that assess children’s symptoms of inattention-overactivity and oppositional-defiant behavior (Volpe et al. 2012). The Oppositional-Defiant scale consists of 5 items, each rated 0 (not at all), 1 (just a little), 2 (pretty much), or 3 (very much). The Oppositional-Defiant scale has demonstrated good internal consistency (α = 0.91) and test-retest reliability (r = 0.69; Waschbusch and Willoughby 2008b) and convergent validity with the externalizing scales on the Child Behavior Checklist (CBCL; Casat et al. 1999) in previous studies. In the present study, internal consistency was acceptable for the Oppositional-Defiant scale (α = 0.81).

Impairment

Overall behavioral impairment was measured using parent ratings on the Impairment Rating Scale (IRS; Fabiano et al. 2006). The IRS consists of 7 items that measure the extent to which a child’s problems affect his or her self-esteem, family functioning, and relationships, and the overall severity of the child’s problems. Items are rated on a visual analogue scale scored from 0 (no problems/no need for treatment) to 6 (severe problems/definitely needs treatment). The item assessing impairment in sibling relationships was not used because not all children had siblings. The mean of the remaining 6 items was computed and used as the overall impairment score. The IRS has demonstrated reliability and validity in previous studies, including a high correlation with criterion measures of impairment (r = −0.79 with the Children’s Global Assessment Scale), stability over 1 year (median r = 0.67), and inter-rater agreement (r = 0.64; Fabiano et al. 2006). Internal consistency for the impairment items was acceptable in the present sample (α = 0.78).

Callous-unemotional traits

A 3-item CU scale (Waschbusch et al. 2004) was used as a measure of CU traits as rated by parents. The CU scale includes the following items: “appears to lack remorse,” “seems to enjoy being mean,” and “is cold or uncaring.” Each item is rated on a 4-point scale of 0 (not at all), 1 (just a little), 2 (pretty much), or 3 (very much). The 3-item CU scale has demonstrated high internal consistency (α = 0.98) and acceptable criterion validity based on its correlation with the Antisocial Process Screening Device (r = 0.60), a well-established measure of CU traits (Waschbusch et al. 2015) in previous studies. Internal consistency for the three CU items was acceptable in the present sample (α = 0.77). The mean score on the 3-item CU scale was used as a dependent variable.

Data Analysis

All analyses were conducted using SPSS version 24 software. Distributions and patterns of missing data were examined prior to conducting analyses. Cross tabulation with a chi square test for independence was used to examine possible gender differences in rates of clinical levels of internalizing symptoms. A 2 × 2 MANOVA with 3 dependent variables (disruptive behavior severity, overall impairment, and CU traits) was computed to examine the main effects of clinical levels of internalizing symptoms and gender, as well as their interaction. Where multivariate main or interaction effects were significant, we followed up with univariate tests. Alpha was set at 0.05 for all analyses. Partial η2 is reported as a measure of effect sizes, with values of 0.01 considered small, 0.06 considered medium, and 0.14 considered large.

Results

Descriptive Statistics

Participants in the present study (N = 284) were drawn from a larger sample of 533 children ages 6–12 who were clinic-referred because of their disruptive behavior. Children with available data for the present study were compared to children without available data in order to determine whether these children differed on key dimensions. There were 104 children in the larger sample who were missing internalizing symptom ratings. Children with and without internalizing symptom ratings did not differ in terms of gender, χ2(1) = 0.42, p = 0.52, or parent-rated impairment (Mann–Whitney U = 19125.0, p = 0.45); however, participants with available internalizing symptom ratings were slightly older (M = 8.8, SD = 1.7) than participants without internalizing symptom ratings (M = 8.5, SD = 2.0; Mann–Whitney U = 19242.0, p = 0.03). An additional 145 participants were missing data on one or more of the dependent variables (oppositionality-defiance, impairment, and/or CU traits). Participants with and without missing data on these dependent variables did not differ in terms of their level of internalizing symptoms, χ2(1) = 3.49, p = 0.06. However, children with complete data on the dependent variables were significantly older (M = 9.2, SD = 1.6) than children missing data on one or more dependent variables (M = 8.2, SD = 1.8), Mann–Whitney U = 22489.5, p < 0.001. Thus, while children in our subsample tended to be slightly older than children in the larger sample, they did not differ in terms of internalizing severity or overall behavioral impairment.

Thirty-seven percent of children were classified as exhibiting clinical levels of internalizing symptoms in our sample. Girls were significantly more likely than boys (51% vs. 33%) to have clinical levels of internalizing symptoms, χ2(1) = 7.65, p = 0.006. Correlations and descriptive statistics for the three dependent variables (oppositionality-defiance, impairment, and CU traits) are presented in Table 1. Weak but significant correlations were found between the dependent variables. The majority of children (68%) were classified as exhibiting clinical levels of ADHD symptoms based on parent ratings on the SDQ Hyperactivity-Inattention subscale. Children with clinical levels of ADHD symptoms were no more likely than other children in our sample to exhibit clinical levels of internalizing symptoms, χ2(1) = 2.46, p = 0.12. In addition, boys and girls were equally likely to exhibit clinical levels of ADHD symptoms, χ2(1) = 3.43, p = 0.06. To ensure that ADHD symptoms were not influencing our results, we re-ran the MANOVA covarying total ADHD symptom ratings. The pattern of results did not change; therefore, we report the results without covarying ADHD symptoms below.

Table 1 Descriptive Statistics and Pearson Correlations between Dependent Variables
Full size table

Multivariate Analysis

Prior to conducting the 2 × 2 MANOVA examining effects of internalizing symptoms and gender on the three dependent variables, we examined our data to determine whether the assumptions necessary for our analysis were met. Significant Kolmogrov-Smirnov and Shapiro-Wilk tests indicated that the distributions of the three dependent variables were significantly different from normal, ps < 0.001. In addition, the covariance of the dependent variables was unequal across groups (Box’s M = 30.546, p = 0.041), and the error variance was not equal across groups for CU (Levene’s F = 3.940, p = 0.009) and impairment (Levene’s F = 2.710, p = 0.045). In order to ensure that our MANOVA results were not attributable to the violations of these assumptions, we re-ran our analyses using nonparametric Mann–Whitney U tests. We found the same pattern of results using the nonparametric and parametric analyses; thus, we report the results of the MANOVA below in order to retain the 2 × 2 multivariate design of our analysis.

Multivariate analyses indicated a statistically significant main effect of internalizing symptoms on the dependent variables, Pillai’s Trace = 0.045, F (3,278) = 4.383, p = 0.005, partial η2 = 0.045, small-to-medium effect size. One way between-subjects effects demonstrated that children with clinical levels of internalizing symptoms had significantly higher parent-rated oppositionality-defiance, callous unemotional traits, and overall behavioral impairment than children with sub-clinical levels of internalizing symptoms (see Table 2).

Table 2 Mean Disruptive Behavior Severity by Level of Internalizing Symptoms
Full size table

Multivariate analyses indicated no statistically significant main effects of gender, and no significant gender by internalizing symptom interactions, on oppositionality-defiance, callous unemotional traits, or overall behavioral impairment, Pillai’s Trace = 0.921, F (3,278) = 0.563, p = 0.636, partial η2 = 0.006, and Pillai’s Trace = 0.007, F (3,278) = 0.687, p = 0.561, partial η2 = 0.007, respectively. See Table 3 for mean scores by gender on the dependent variables.

Table 3 Mean Disruptive Behavior Severity by Gender
Full size table

Discussion

This study investigated whether the presence of comorbid internalizing symptoms is associated with differences in the severity of disruptive behavior among clinic-referred children with DBD, in terms of their level of oppositionality-defiance, overall behavioral impairment, and the presence of CU traits. A considerable proportion (37%) of children in our sample had clinical levels of internalizing symptoms based on parent report, consistent with rates of comorbid anxiety and depression reported previously in a large, clinic-referred sample of children with disruptive behavior (Greene et al. 2002). As hypothesized, children with DBD and clinically elevated internalizing symptoms had higher parent-reported levels of oppositional-defiant behavior and overall behavioral impairment than children with sub-clinical levels of internalizing symptoms, with small-to-medium effect sizes. In addition, and unexpectedly, children with clinically elevated internalizing symptoms also had higher levels of CU traits, also with a small-to-medium effect size. Our results suggest that children with DBD and comorbid internalizing symptoms are an especially symptomatic and impaired group. Below we discuss our findings in the context of previous research on internalizing-externalizing comorbidity as well as potential clinical implications.

Previous evidence and theory suggest that a dual pathway may exist, whereby comorbid internalizing symptoms either mitigate (buffer hypothesis) or exacerbate (multiple problems hypothesis) the severity of disruptive behavior (Drabick et al. 2010). We did not find support for the buffer hypothesis in our sample. Instead, our results support the multiple problems hypothesis. Consistent with several previous studies (Biederman et al. 1991; Cunningham et al. 2013; Garai et al. 2009; Humphreys et al. 2012), children with clinical levels of internalizing symptoms showed more severe levels of disruptive behavior and behavioral impairment. These results are also in line with broader findings that difficulties with emotion regulation exacerbate the negative disposition of children with ODD, reduce self-regulation, increase the likelihood of impulsivity and anger, and compromise children’s ability to control their behavior (Muris and Ollendick 2005; Zeman et al. 2006).

Unexpectedly children with clinical levels of internalizing symptoms also showed significantly higher levels of CU traits than did children with sub-clinical internalizing symptoms. Past research and theory have suggested that youth with high levels of CU traits are unlikely to exhibit comorbid internalizing symptoms (Frick and White 2008; Hawes and Dadds 2005). Instead, children and youth with high levels of CU traits have been characterized as exhibiting a fearlessness that would be inconsistent with an internalizing clinical presentation (Frick and Morris 2004). However, we speculate on a few reasons for the discrepant findings of the present study. First, emerging evidence suggests that some youth present with a combination of CU traits and internalizing symptoms, particularly anxiety, which is often referred to as a “secondary variant” of callous-unemotional traits (Goulter et al. 2017; Kahn et al. 2013). Similarly, Meehan et al. 2017 found that a subgroup of youth who displayed interpersonal callousness, and demonstrated high levels of co-occurring anxiety, also presented with greater levels of additional psychopathology such as ADHD, CD and ODD (Meehan et al. 2017). Second, pre-adolescent children in our clinical sample were between 6 and 12 years of age. It is possible that internalizing symptoms in these children when expressed behaviorally appear callous or unemotional. For example, a sensitive and withdrawn child may present as indifferent to the social and emotional needs of peers; however, this indifference may be associated with underlying anxiety and social aversion. Third, although children in this clinical sample showed elevated disruptive behavior, they did not show high levels of CU (i.e., if compared to other clinical and forensic samples of older youth in other studies). As such, internalizing symptoms and CU traits may have a stronger association at lower levels of CU severity. Overall, our results are consistent with the multiple problems hypothesis, given that CU traits are associated with a more persistent and severe course of disruptive behavior (Frick et al. 2014). These findings underscore the possibility that CU traits and internalizing symptoms do co-occur in some children with disruptive behavior; however, our assertions are in need of further evaluation with more comprehensive measures of CU traits and internalizing symptoms in other clinical samples of children.

There were no gender differences in our sample in terms of the severity of children’s oppositional-defiant behavior, behavioral impairment, or CU traits. However, consistent with rates of internalizing disorders in previous epidemiological and disruptive behavior samples (Costello et al. 2003; Loeber et al. 2000; Maughan et al. 2004; Merikangas et al. 2010; Waschbusch 2002), girls in our sample were more likely than boys to have clinical levels of internalizing symptoms. Aside from this higher rate of heterotypic comorbidity, we did not find other evidence of the gender paradox, whereby girls with disruptive behavior have more severe clinical presentations than boys with disruptive behavior (Loeber et al. 2000). In addition, the effects of internalizing symptoms on the severity of disruptive behavior were consistent across boys and girls in our sample. Our results are in line with previous findings that gender does not influence the severity of CD or ODD (Loeber et al. 2000; Maughan et al. 2004; Romano et al. 2001; Simonoff et al. 1997).

Limitations and Future Directions

Results of the present study must be considered in light of its limitations. First, we used relatively brief measures of the constructs studied (internalizing, oppositionality-defiance, and CU traits). Second, we relied on parent report only. It is possible that informant effects may have increased the associations between variables (namely internalizing and disruptive behavior severity) in our sample. Moreover, as internalizing problems are less observable to parents than externalizing behavior (De Los Reyes et al. 2015), it is possible that parents may have underreported children’s internalizing symptoms. Future studies would benefit from the use of more comprehensive multi-informant measures of internalizing and externalizing symptoms. In addition, although anxiety and depression show high rates of overlap in children, future investigations using separate measures of anxiety and depressive symptoms may help to disentangle their relative importance for disruptive behavior severity. Third, our study is cross-sectional and thus we are unable to infer causal relationships between internalizing symptoms and the severity of children’s disruptive behavior. Additional research incorporating longitudinal components is necessary to provide information on the direction of the association between internalizing symptoms and disruptive behavior. Finally, the results are based on a clinic-referred sample, and participants were relatively homogeneous in terms of ethnicity and parent education. Replication in community samples and in samples with more diverse representation of ethnicities and socioeconomic statuses is needed to increase the generalizability of the results.

Clinical Implications

Our results suggest that many children accessing mental health services for disruptive behavior also experience comorbid internalizing symptoms. Thus, it is important for clinicians and health providers to assess for the presence of internalizing symptoms when children present with disruptive behavior. This is particularly true given that our results suggest children with this pattern of heterotypic comorbidity exhibit the most severe and impairing levels of disruptive behavior. The presence of comorbid internalizing symptoms may also inform treatment decisions. Internalizing symptoms do not appear to reduce the effectiveness of psychosocial treatment for disruptive behavior (Aitken et al. 2018; Chase and Eyberg 2008; Masi et al. 2013); however, there is some evidence that children with disruptive behavior who have lower levels of internalizing respond better to treatment when a parent component is included, whereas those with higher levels of internalizing benefit equally from parent-, child-, or teacher-focused interventions (Beauchaine et al. 2005). Further research is needed to determine the extent to which treatments for disruptive behavior may also be associated with reductions in internalizing symptoms, and vice versa. It may also be possible to target internalizing and externalizing symptoms through the use of modularized treatment approaches (Chorpita and Weisz 2009).

Summary

Clinic-referred children with disruptive behavior who also presented with elevated levels of internalizing symptoms demonstrated more oppositionality-defiance, callous-unemotional traits, and overall behavioral impairment. While girls were more likely than boys to have clinical levels of internalizing symptoms, no significant effects of gender with regards to severity of disruptive behavior were found, in contrast to the notion of a gender paradox. Moreover, the effects of internalizing symptoms on the severity of disruptive behavior did not differ between boys and girls. The findings of this study support the multiple problems hypothesis, as well as the clinical utility of considering comorbid internalizing symptoms in children with DBDs within the context of assessment and intervention planning.