Introduction

Cognitive factors have increasingly been recognized as playing a role in the etiology and maintenance of child anxiety disorders (Vasey and MacLeod 2001). However, few measures have been developed for assessing cognitions specific to anxious children. Schniering and Rapee (2002) note that of the few existing cognitive measures for children, nearly all are downward extensions of measures initially developed for adults. Consequently, these authors designed a self-report measure of negative self-statements, entitled the Children’s Automatic Thoughts Scale (CATS), which was intended to be developmentally sensitive to children as well as inclusive of specific cognitions associated with both internalizing and externalizing disorders (Schniering and Rapee 2002). Using a community sample of 350 boys and 628 girls (7–16 years old), the authors conducted a confirmatory factor analysis of the CATS, which revealed four subscales: thoughts associated with Physical Threat, Social Threat, Personal Failure, and Hostility (Schniering and Rapee 2004a).

The authors of the CATS have found that it has high internal consistency and good test–retest reliability in addition to good discriminant validity among anxious, depressed, externalizing, and community groups of children in Australia (Schniering and Rapee 2002, 2004b). In addition, Schniering and Lyneham (2007) recently evaluated the measure’s utility in a large clinical sample of 460 boys and 431 girls with primary anxiety disorders (ages 7–17). They found that overall internal consistency of the subscales was high (0.82–0.92), and convergent validity with relevant symptom measures was strong. The CATS was sensitive to effects of treatment, and divergent validity was moderate, as shown by significant differences between anxiety-only, anxiety + depression, and anxiety + oppositional diagnostic groups. Notably, self-report of depression was strongly associated with the Physical Threat and Social Threat subscales (in addition to the Personal Failure subscale), with these correlations equaling or exceeding those between the threat subscales and the Revised Children’s Manifest Anxiety Scale (RCMAS; Schniering and Lyneham 2007). On the other hand, Schniering and Rapee (2004b) found evidence for the specificity of the CATS: Personal Failure was the strongest predictor of depressive symptoms on the Children’s Depression Inventory (CDI), while Social Threat accounted for the most variance in anxious symptoms on the RCMAS. Notably, this sample included 79 girls and 81 boys with a broad range of DSM-IV disorders, including depression, and 97 girls and 103 boys (ages 7–16) from the community.

The development of valid assessment measures is essential to the identification of children with emotional disorders who could benefit from early intervention. The CATS is a potentially valuable measure of children’s cognition in both research and clinical settings. In particular, the CATS could be very useful in monitoring changes in maladaptive cognitions over the course of cognitive therapy or other treatments for childhood disorders. However, no studies to date have validated the measure against specific syndromes derived from the DSM-IV (American Psychiatric Association 2000). In addition, all studies of the measure thus far have been conducted using samples of children and adolescents in Australia. Thus, the purpose of the present study was to provide a preliminary evaluation of the validity and specificity of the CATS in a sample of anxious and non-clinical children in the United States.

Methods

Participants

A total of 80 children (ages 7–14) participated in this study. Forty children (18 boys, 22 girls) with a mean age of 10.65 years (SD = 2.20), were included in the clinically anxious sample and recruited from consecutive referrals to an anxiety disorders clinic. These children were almost exclusively Caucasian (97.7%). All children were administered a diagnostic interview (the Anxiety Disorders Interview Schedule for the DSM-IV, Child and Parent Versions, Silverman and Albano 1996) as part of their initial assessment and diagnosed with a principal anxiety disorder (see Table 1). Of the children who met criteria for a clinical anxiety disorder, 15 (37.6%) and 8 (20%) met criteria for one and two additional clinical anxiety disorders, respectively. Three children (7.6%) met criteria for an additional clinical depressive disorder (Major Depression or Depressive Disorder Not Otherwise Specified).

Table 1 Principal diagnoses of clinical participants (n = 40)
Full size table

Forty additional children from the community (19 boys, 21 girls), with a mean age of 11.30 years (SD = 2.04), were recruited to participate in the study from advertisements posted on community and Internet bulletin boards. Children were primarily Caucasian (75%), African American (12.5%), and Asian American (5%). Because of time constraints, the ADIS-IV-C/P was not administered to the non-clinical sample. However, in order to participate in the study, the non-clinical children could not have previously been diagnosed with an anxiety or mood disorder, and they had to score at or below the mean (T = 50) on a self-report measure of anxiety and depression (the Revised Children’s Anxiety and Depression Scale, see below). The rationale for this conservative inclusion criterion was to increase the likelihood that children in the control group did not have an anxiety or depressive disorder.

Measures

Children’s Automatic Thoughts Scale (CATS; Schniering and Rapee 2002). The CATS consists of 40 negative self-statements; for each statement, children rate on a scale from 0 to 4 how often they experienced the self-statement in the past week. The four subscales include: Physical Threat (e.g., “I’m going to get hurt”), Social Threat (e.g., “People are thinking bad things about me”), Personal Failure (e.g., “I can’t do anything right), and Hostility (e.g., “Some people deserve what they get”).

Revised Child Anxiety and Depression Scale (RCADS; Chorpita et al. 2000). The RCADS is a 47-item self-report measure that contains six factors, which correspond to five DSM-IV-TR anxiety disorders (excluding specific phobia and PTSD) and major depressive disorder. Internal consistency of the subscales ranges from 0.73 (OCD) to 0.82 (social phobia). Test–retest reliability over a 1 week period is adequate to good, ranging from 0.65 (OCD) to 0.80 (social phobia). Discriminant, convergent, and factorial validity were supported in both a community sample (Chorpita et al. 2000) and a clinical sample (Chorpita et al. 2005). Norms are available by gender and grade for children in grades three through 12. A T-score of 65 and above for each scale is considered to be in the “clinical” range.

Procedure

The Institutional Review Board at Boston University approved all study procedures. Informed consent/assent forms were signed by mothers and their children. Children and their mothers then completed the questionnaires as part of a larger study of cognitive biases in anxious children (Micco and Ehrenreich 2008). At the study’s completion, participants were given $40 or a free treatment session at the clinic, and children were given a small prize.

Data Analysis

The Statistical Package for Social Sciences (SPSS 12.0; SPSS Inc. 2004) was used to conduct data analyses. Internal validity was examined by: (1) calculating internal consistency of each subscale using Cronbach’s α, and (2) calculating Pearson’s correlations between each item of each subscale (i.e., inter-item correlations). For criterion validity, we first conducted an ANOVA to examine mean differences in subscale scores between the clinical and non-clinical samples. We then completed a discriminant function analysis to determine the percentage of cases correctly classified into “clinical” and “non-clinical” groups by the CATS; the total score on the CATS was the independent variable and clinical status was the dependent variable. To examine convergent validity, we calculated partial correlations between the CATS subscales and the RCADS subscales. Because age and gender have previously been found to be associated with scores on the CATS (Schniering and Lyneham 2007), we controlled for these variables in the partial correlations, in addition to clinical status.

Results

Internal Validity

Internal consistency of the subscales was generally high, with Cronbach’s α of 0.95 (Social Threat), 0.93 (Personal Failure), 0.91 (Physical Threat), and 0.83 (Hostility). Inter-item correlations for each subscale were moderate to high: 0.62–0.88 for Physical Threat, 0.60–0.91 for Social Threat, 0.70–0.89 for Personal Failure, and 0.35–0.79 for Hostility; all inter-item correlations were statistically significant at P < 0.001. Data on inter-item correlations for each item of the CATS are available upon request from the first author.

Criterion Validity

Mean subscale scores for each group are listed in Table 2. The anxious group had significantly higher scores on the Physical Threat, Social Threat, and Personal Failure subscales. Results of the discriminant function analysis showed that of the total sample, 90% were classified correctly, with 87.5 and 92.5% of the clinically anxious and non-clinical samples correctly classified, respectively; these results approached statistical significance (Wilks’ Lambda = 0.40, P = 0.09), although a larger sample size would likely have led to statistical significance.

Table 2 CATS subscale scores: clinical versus non-clinical samples
Full size table

Convergent Validity

Controlling for age, gender, and clinical status, we found that the Physical Threat subscale of the CATS was significantly correlated with the Generalized Anxiety Disorder and Separation Anxiety Disorder (SAD) subscales of the RCADS. The Social Threat subscale was highly correlated with the Social Anxiety Disorder (SOC) subscale of the RCADS, and also significantly correlated with the Major Depression (MDD) and SAD subscales. Cognitions associated with Personal Failure on the CATS were significantly correlated with both the MDD and SOC subscales of the RCADS. Results are summarized in Table 3.

Table 3 Partial correlations controlling for age, sex, and clinical status
Full size table

Discussion

The main results of this study show that in a sample of clinically anxious and non-clinical children: (1) the CATS subscales have good internal consistency, (2) there were significant differences between groups in the expected direction on the Physical Threat, Social Threat, and Personal Failure subscales, and (3) the CATS (total score) accurately classified children into the anxious versus non-clinical groups 90% of the time, a result that approached statistical significance. In support of the measure’s convergent validity with the RCADS scales, after controlling for participants’ age, sex, and clinical status, we found that: (1) Physical Threat was most highly correlated with self-report of Generalized Anxiety Disorder, followed by SAD, (2) Social Threat was strongly correlated with self-report of SOC, followed by MDD, and (3) Personal Failure was correlated with MDD and SOC. We did not find a significant relationship between the Hostility subscale and any of the RCADS scales.

Overall, the internal, criterion, and convergent validity of the CATS was supported in this sample, consistent with findings of Schniering and Lyneham (2007), whose sample was similarly comprised of children with primary anxiety disorders. Of interest, however, is the nearly equal magnitude of the association between the Personal Failure subscale on one hand and the Social Anxiety and Major Depression subscales of the RCADS on the other. In addition, while SOC was most strongly associated with the Social Threat subscale, MDD was also moderately correlated with this subscale, similar to findings of Schniering and Lyneham using the CDI.

Beck’s (Beck and Clark 1988) cognitive specificity theory (1976) argues that there are cognitive biases specific to different emotional disorders. However, research on cognitive biases in children and adolescents has suggested that while there are distinctions in cognition across broadband disorders (e.g., internalizing versus externalizing; Epkins 2000; Leung and Wong 1998), often there are similarities in cognitive biases for disorders within each broadband category (Epkins 1996; Treadwell and Kendall 1996). In other words, childhood anxiety and depressive disorders may share a common underlying cognitive vulnerability. Indeed, our finding of a significant overlap between cognitions associated with social anxiety and major depression in children is consistent with previous findings of similarities in cognitive content between youth with social anxiety and dysphoria (Epkins 1996).

We did not find an association between the Hostility subscale and any of the RCADS subscales; this is not a surprising finding, given that the RCADS does not contain a subscale that measures oppositional or aggressive behavior. Future studies should employ a self-report measure of externalizing symptoms in order to confirm the relationship between Hostile thoughts and oppositional behavior or aggression.

Our study supports the validity of the CATS as a measure of maladaptive cognition in children. As such, this measure could be extremely useful in assessing cognitive patterns associated with different childhood disorders and in monitoring changes in cognition over the course of treatment. However, it is important to note that the CATS is not intended to be a diagnostic measure on its own; rather, the measure should serve as an adjunct to a careful diagnostic and functional assessment for childhood psychiatric disorders.

Our results are tempered by several limitations. First, our small sample size and consequent reduced statistical power rend these results as preliminary and may have led us to miss findings with smaller effect sizes; in particular, trend results may have become significant with sufficient statistical power. Second, our study lacked a clinical control group to compare differences in cognition among children with a wider range of psychiatric disorders. Further, though we excluded children previously diagnosed with anxiety or depression from our control group and employed a conservative cut-off score on the RCADS to be included in the control group, we cannot be certain these participants did not meet criteria for a disorder because they were not administered a diagnostic interview. Third, our sample was limited to children 14 years old and younger; thus, we cannot draw conclusions based on this study about the utility of the CATS with older adolescents, although we are now in the process of collecting the CATS in a study of adolescents. Thus, future studies examining the psychometric properties of this measure should include a larger and more heterogenous group of children and adolescents.

Nevertheless, this study is the first to validate the CATS against specific syndromes derived from the DSM-IV (American Psychiatric Association 2000) in a clinical and non-clinical sample of children living in the United States. In sum, our findings are consistent with those of the Australian researchers who developed the measure (Schniering and Rapee 2002, 2004a, b; Schniering and Lyneham 2007), and suggest that the measure is a valid method for assessing anxious and depressive cognitions among children with primary anxiety disorders.