Erroneous belief systems or patterns of thought, such as cognitive bias, contribute to cognitive vulnerability that predisposes an individual to psychopathology (Riskind and Black 2005). Theoretical and empirical data suggest that in the presence of cognitive vulnerability, the occurrence of a negative event will trigger a pattern of negatively biased information processing that initiates a downward spiral into depressive or anxiety symptoms (Lakdawalla et al. 2007; Weems and Watts 2005). Cognitive vulnerability is proposed to be an important factor in the development and maintenance of childhood psychopathology, especially depressive and anxiety symptoms (e.g., Kingery et al. 2009; Michael and Karen 2014; Ishikawa 2012; Kempton et al. 1994; Weems et al. 2007; Yurica and DiTomasso 2005). According to various cognitive theories, several distinct cognitive vulnerability factors are hypothesized to be present in children and adolescents, namely dysfunctional attitudes, negative cognitive style, or a ruminative response style (Lakdawalla et al. 2007).

As part of a negative belief system (Beck et al. 1979), distorted cognitive processing involves systematic misinterpretations of new information, which is found to be particularly influential in the development and maintenance of psychopathology in youth (e.g., Ishikawa 2012; Kempton et al. 1994; Weems et al. 2007; Yurica and DiTomasso 2005). Distorted cognitive processing in youth is represented by various cognitive errors such as selective abstraction, overgeneralization, personalization, and others (e.g., Joiner and Wagner 1995; Kingery et al. 2009; Leitenberg et al. 1986). Accumulated evidence indicate that cognitive errors are not only associated with anxiety and depressive symptoms (Curry et al. 2006; Kingery et al. 2009; Leitenberg et al. 1986; Pereira et al. 2012; Silverman et al. 1999; Watts and Weems 2006; Weems et al. 2007), but also possibly with externalizing symptoms such as disruptive behaviors, as well as comorbid depression/anxiety and externalizing symptoms (e.g., Epkins 2000; Kempton et al. 1994; Schepman et al. 2014).

Much of what is currently known about how cognitive errors operate in depressive and anxiety symptoms among children and adolescents are based on the Children’s Negative Cognitive Error Questionnaire (CNCEQ; Leitenberg et al. 1986). The original CNCEQ measures four cognitive errors: catastrophizing, overgeneralizing, personalizing, and selective abstraction. The questionnaire was recently revised into a scale with five cognitive errors: underestimation of the ability to cope (i.e., the tendency to judge oneself as unable to cope with potentially threatening situations), personalizing without mind reading (i.e., taking responsibility for events, but not perceiving threat, without evidence of personal causality), mind reading (i.e., concluding that someone is reacting negatively or thinking negatively toward oneself, without specific evidence to support that conclusion), selective abstraction (i.e., allowing one negative aspect of a situation to colour the whole), and overgeneralizing (i.e., formulating general rules on the basis of limited experience and applying them too broadly; Maric et al. 2011).

From a clinical and psychometrical point of view, the extent to which the CNCEQ and its revision are able to capture cognitive errors in children and adolescents has yet to be established due to gaps in research. For example, distinct cognitive vulnerability factors, including cognitive errors, appear to contribute to expression of depression and anxiety disorders through different mechanisms (for review see Lakdawalla et al. 2007; Weems and Watts 2005). For example, there are data indicating that cognitive errors might have direct influence on anxiety disorder expression (Weems et al. 2001) or a mediating role when considered with attributional biases in depression (Cole and Turner 1993).

Another gap in research is the inconsistency in findings in terms of which cognitive errors are specific to anxiety disorders or to depression, and which ones are common to both (e.g., Epkins 1996; Weems et al. 2001). While Schwartz and Maric (2015) reported the error “overgeneralizing” as a predictor of depression, some authors reported the error as predictor of both anxiety and depressive symptoms (Epkins 1996; Weems et al. 2001). The same applies for the error “personalization” and “underestimation of ability to cope”, which was argued to be predictive of both types of symptoms (Weems et al. 2001; Maric et al. 2011).

There is evidence from factor analytic studies using the CNCEQ to support the original factor structure of distinct cognitive errors, but data are inconsistent due to limited analyses in clinical samples. The first factor analytic study using a confirmatory approach with data from a community sample supported a four-factor model including one general factor and three factors representing specific content areas (i.e., social, academic, and athletic; Cole and Turner 1993). This finding was consistent with another study using data from clinically depressed youth, which also supported a model with the original four factors (Kingery et al. 2009). Another study, again using a confirmatory approach on data from the general population, supported a model with three factors corresponding to catastrophizing, personalizing, and selective abstraction (Karakaya et al. 2007). Finally, the results of a study using an exploratory approach with data from a clinical sample supported a single-factor solution as a global construct indicative of “negative thinking” (Messer et al. 1994). A single factor was also suggested in a study combining data from community samples of adolescents in the United Sates and Hong Kong (Stewart et al. 2004). In an attempt to overcome the above inconsistency, a five factor model was suggested using a further reduced list of items (i.e., 16 items; Maric et al. 2011). Nevertheless, the most recent study using principal component analysis indicated that 11 items loaded on the proposed five components, while five items cross-loaded onto other factors that were initially deemed to be unrelated (Schwartz and Maric 2015).

Finally, cross-cultural data on the measurement aspects of the CNCEQ are scarce. The original CNCEQ was developed in the Unites States on an English speaking sample and it is available in Chinese for Hong Kong (Leung and Wong 1998) and Turkish (Karakaya et al. 2007). The CNCEQ-R was developed from a community sample of Dutch children and adolescents (Maric et al. 2011). Using the original and Chinese version, both having the same factor model, it was observed that cognitive errors were lower among adolescents in Hong Kong than in the United States (Stewart et al. 2004). While this finding could represent true differences, it could also be an effect of measurement non-invariance of the questionnaire. For the Turkish version (Karakaya et al. 2007), the validation study supported a factor model including three cognitive errors (i.e., catastrophizing, personalizing, and selective abstraction) and not four as in the original. The version was also found to be more strongly correlated to depression than to anxiety (Karakaya et al. 2007). Considering all of the above, there is a need to further examine the factor structure of the CNCEQ and its revision, as well as how these factors relate to anxiety and depression symptoms among children and adolescents, including those from clinical samples and different cultural groups. Of particular importance is to evaluate the CNCEQ-R, given that available psychometric data from two studies indicate that it could better serve in the conceptualization and assessment of distorted cognitive processing in youth than its predecessor (Maric et al. 2011; Schwartz and Maric 2015).

The present study had four aims. The first aim was to evaluate the factor structure of the CNCEQ-R in a community and clinic-referred sample of Serbian children and adolescents. The second aim was to test the measurement invariance of the CNCEQ-R across age, gender, and populations (clinical–non-clinical), as this aspect of validity has not been previously evaluated. A prerequisite for cross-group comparisons is that the theoretical construct is measured in each group in the same way, namely that construct equivalence is achieved for the scale representing that the theoretical construct when tested simultaneously across several groups (Dimitrov 2010). Therefore, in order to compare the presence of cognitive errors across various groups, an important aspect that needs to be demonstrated is that the factorial structure of the CNCEQ-R is invariant (i.e., possess measurement invariance; Byrne and Watkins 2003). The third aim was to evaluate how cognitive errors measured with the CNCEQ-R relate to anxiety and depressive symptoms in general. Finally, the fourth aim was to compare levels of cognitive errors present among children and adolescents with anxiety disorders, depression, or externalizing disorders.

Methods

Participants

Data for the present study were collected from a community and clinical sample of children and adolescents. Participants in the community sample were from sixth to eighth grades and were randomly selected from two public elementary schools in Northern Serbia. The clinical sample included children and adolescents who were referred to a public child specialist clinic for a psychiatric assessment. At the time of assessment, none of the children and adolescents were previously diagnosed as having a psychiatric disorder, nor were they on any ongoing treatment. The assessments and diagnoses were made according to the Diagnostic and Statistical Manual of Mental Disorders 5th edition (DSM–5; American Psychiatric Association 2013).

In the community sample, data were available for 257 participants, with 114 (44.4 %) boys and 143 (55.6 %) girls. There were 66 (25.7 %) children aged 12 years and 191 (74.3 %) adolescents aged 13–15 years. The mean age was 13.09 (SD = 0.79) years. The cut-off of 12 was used in the present study, as it was used previously with the CNECQ (Karakaya et al. 2007, Schwartz and Maric 2015). For the clinical sample, data were available for 198 participants, with 92 (46.5 %) boys and 106 (53.5 %) girls, of which 55 (27.8 %) children aged 9–12 years and 143 (72.2 %) adolescents aged 13–18 years. The mean age was 13.97 (SD = 2.27) years. Table 1 provides the diagnoses for this sample. One to two co-morbid disorders were present in 37 (18.7 %) children and adolescents, while among 15 (7.6 %) no specific psychopathology was identified.

Table 1 Primary DSM diagnoses among the clinic-referred youth sample (N = 198)
Full size table

Questionnaires

CNCEQ-R. The CNCEQ-R is a self-repot questionnaire for assessing cognitive errors in 9–17 year-olds. It contains 16 items in five subscales. The subscales measure the errors “underestimation of the ability to cope” (three items), “personalizing without mind reading” (three items), “mind reading” (four items), “selective abstraction” (three items), and “overgeneralizing” (three items). Each item consists of a description of a situation and of the thought someone might have in that situation. The child indicates on a 5-point scale how similarly he/she would think in this situation, ranging from “almost exactly like I would think” (5 points) to “not at all like I would think” (1 point). The CNCEQ-R total score ranges from 16 to 80, with higher scores indicating more distorted cognitive processes. The original study reported high test–retest reliability of the total questionnaire score (r = 0.90) and moderate to high for the subscales (range 0.71–0.85). The internal consistency of the total score was also good (α = 0.80), but the alphas for the five subscales were low (all below 0.62; Maric et al. 2011).

Revised Child Anxiety and Depression Scale (RCADS). The RCADS is a 47-item self-report questionnaire about depressive and anxiety symptoms in youths (Chorpita et al. 2000). The respondent indicates on a 4-point scale how often each symptom is present ranging from “never” (0 point) to “always” (3 points). The RCADS depression and anxiety score are calculated as the sum of all answered items, with higher scores indicating more frequent symptoms. Psychometric studies using the RCADS across different samples consistently report adequate internal consistency and test–retest reliability for the overall scale and its subscales, with satisfactory convergent/discriminant validity (e.g., Chorpita et al. 2000, 2005; de Ross et al. 2002). In the present study, Cronbach’s coefficients for the RCADS anxiety and depression scores were 0.95 and 0.87, respectively.

Both questionnaires were professionally translated into Serbian following the same procedure of two forward translations, a single form development, a single back-translation, and the harmonization of the translations with the originals. This research group pre-tested the translations in semi-structured interviews with a group of 10 children (age range 9–15 years; four of which were boys). Four separate semi-structured interviews were organized to explore clarity, comprehensibility, and response process for every item on the questionnaires (cognitive debriefing). Finally, the content, face validity, and equivalence with the originals (conceptual, item, semantic, and operational) of the translations were explored. All items were considered to be comprehensive, precise, and relevant for assessing intended constructs in the translations, thus they were unchanged and no item was added, replaced or omitted.

Statistical Analysis

Data from both samples were combined for statistical analyses. All data were combined because cognitive errors and psychological symptoms themselves are conceptualized along a continuum from none to most severe. In addition, this study intended to capture the widest possible variability in cognitive errors present among children and adolescents. Confirmatory factor analysis (CFA) was first carried out using maximum likelihood solution with the AMOS version 7 software (Arbuckle 2006). The fit of the model to the data was evaluated using the following fit indices and their recommended benchmarks (Kline 2005): χ2/df ratio (<3 good), the comparative fit index—CFI (>0.90 acceptable, >0.95 excellent), the goodness of fit index—GFI (>0.90 acceptable, >0.95 excellent), and root mean square error of approximation—RMSEA (<0.08 acceptable, <0.06 excellent). Afterwards, the model was tested for measurement invariance by gender (boys vs. girls), age (children aged 8–12 years old vs. adolescents aged 13–18 years old), and population (clinically-referred vs. non-referred). Afterwards, multigroup CFA (MG-CFA) was carried out to test measurement invariance. Several types of measurement invariance form a nested hierarchy (i.e., dimensional, configural, metric, and scalar), which is required to allow latent means comparisons across groups (Byrne and Watkins 2003). When testing one scale across different groups, dimensional invariance means that the same number of common factors is present; configural invariance means that the same items for each group are associated with the same factors; metric (i.e., weak measurement) invariance means that the common factors have the same meaning; scalar (i.e., strong measurement) invariance means that the intercepts or threshold of the items are equivalent. Evidence of invariance was based on nested model testing using a CFI (ΔCFI ≤ 0.01) and χ2 (p > .01) difference (French and Finch 2006).

Internal consistency (Cronbach’s alpha), test–retest reliability, and correlations between the CNCEQ-R subscales (Pearson correlation coefficient) were then calculated. Test–retest reliability analysis involved 30 participants from the clinical sample who completed the questionnaires again one week after the first testing.

In order to determine which cognitive errors were predicative of anxiety or depressive symptoms, hierarchical regression analyses were conducted. All five CNCEQ-R scales were entered separately as predictors of the RCADS anxiety and RCADS depression scores. The final analyses assessed how the CNCEQ-R cognitive errors differ among groups of children and adolescents with clear clinical diagnosis based on the DSM 5 (American Psychiatric Association 2013).

Results

The CFA results indicated an overall good fit to the data with χ2/df = 2.164, CFI = 0.938, GFI = 0.948, and RMSEA = 0.051. This five factor model produced superior levels of fit compared to a one-factor model, which had fit indices of χ2/df = 3.339, CFI = 0.862, GFI = 0.902, and RMSEA = 0.072. The results of MG-CFA indicated that the five-factor model was fully invariant across the studied groups (Table 2). However, the results of MG-CFA showed that the CFI (0.853) and GFI (0.856) were slightly below acceptable values to claim good model fit for children. Similarly, the CFI (0.892) for non-referred youth was also below levels of acceptable fit. Table 3 provides all CNCEQ-R scores for tested groups.

Table 2 Multi-group confirmatory factor analyses of the CNCEQ-R
Full size table
Table 3 CNCEQ-R scores among different groups
Full size table

The correlations between the CNCEQ-R scales were all positive and were in the medium range (Table 4). Cronbach’s alpha for the total score was 0.86, while the alphas for the subscales ranged from 0.54 to 0.66 (Table 4). In terms of test–retest reliability, Pearson’s correlation coefficient for the total score was 0.83 and for the subscales ranged from 0.65 to 0.83 (Table 4).

Table 4 Pearson correlations, Cronbach’s’ s alpha, and test–retest r for the five CNCEQ-R subscales
Full size table

The results of hierarchical regressions for the RCADS depression score showed that the correlation coefficient of the model was significantly different from zero, F (5, 451) = 24.59, p < .001, with R2 = 0.22, implying that the cognitive errors explained 22 % of the variance in depressive symptoms (Table 5). The error “overgeneralizing” contributed the most to this variance, and “underestimation of ability to cope” also appeared as a significant predictor. In the model for RCADS anxiety, the correlation coefficient of the model was also significantly different from zero, F (5, 419) = 35.71, p < .001, with R2 = 0.29, implying that the cognitive errors explained 29 % of the variance in anxiety symptoms. The error “mind reading” contributed the most to the variance, while “underestimation of ability to cope”, “overgeneralizing”and “personalizing without mind reading” were also significant predictors. In both models, the cognitive error “selective abstraction” did not appear as a significant predictor, but it showed an inverse correlation with the RCADS anxiety score.

Table 5 Results of hierarchical regression analyzes (N = 451)
Full size table

It was possible to create four different groups with clear DSM diagnosis for the final analyses: youth with an anxiety disorder only, youth with depression only, youth with comorbid anxiety disorders and depression, and youth with externalizing disorders only (i.e., conduct disorder and/or attention deficit/hyperactivity disorder). Among the groups, children and adolescents differed significantly in age (F (df) = 3.12 (3), p = .003) and gender (χ2 (df) = 14.73 (3), p = .002), thus the two variables were entered as covariates in analysis of variance (Table 6). The results indicated that the CNCEQ-R total score was significantly higher in youth with depression only than in youth with anxiety disorders onlyor youth with externalizing disorders only. In terms of cognitive errors, significant differences among the groups were only observed for the “overgeneralizing” and “mind reading” scores.

Table 6 CNCEQ-R scores among clinical groups (N = 115)
Full size table

Discussion

The CNCEQ (Leitenberg et al. 1986), which was designed to assess distorted cognitive processing in youth, was recently revised (Maric et al. 2011; Schwartz and Maric 2015) and the present study evaluated its validity in a large sample of non-referred and clinically-referred youth. The study supported the suggested five-factor structure of the revised version (Maric et al. 2011). The fit indexes indicated acceptable fit of the data to the model across age, gender, and population samples, with the exception that the CFI and GFI values that were slightly below the set values for children and non-referred youth. These lower values may be an effect of the sample itself or its size, considering that the adolescent sample was almost three times larger than the child sample (Fan et al. 1999). The five-factor model was also found to be fully invariant between boys and girls, children and adolescents, as well as between clinically-referred and non-referred youths. This implies that the latent CNCEQ-R factors are probably associated with the same items and have the same meanings across the groups, and that the comparisons of the CNCEQ-R means across these groups are meaningful (French and Finch 2006).

Results from further analyses were in line with the normative data (Maric et al. 2011). The correlations among the CNCEQ-R scales were medium, suggesting that the five cognitive errors are part of a higher construct. The internal consistency for the five scales scores was low, and only the total score yielded a high alpha. This particular finding, although possible to explain by a small number of items per scale, indicates that the retained items have low homogeneity in measuring suggested cognitive errors. Nevertheless, the test–retest reliability of the CNCEQ-R was found to be sufficient for all subscales, except for the cognitive error “selective abstraction”, which also had low test–retest reliability in the original study (Maric et al. 2011).

We investigated how cognitive errors predicted anxiety and depressive symptoms in general. The total variance in depressive symptoms explained by the CNCEQ-R was 22 %, which is a larger amount than previously reported (16 % in Schwartz and Maric 2015), but smaller than reported in studies with the original CNCEQ (e.g., 24 % in Weems et al. 2001 or 30 % in Weems et al. 2007). On the other hand, the total variance in anxiety symptoms explained by the CNCEQ-R was 29 %, which is similar to the previous study (30 % in Schwartz and Maric 2015), and greater than amounts reported in studies with the original measure (e.g., 20 % in Weems et al. 2001 or 12 % in Weems et al. 2007). Considering the cognitive errors separately, depressive symptoms were found to be predicted only by “underestimation of ability to cope” and “overgeneralizing”, while anxiety symptoms were additionally predicted by “personalizing without mind reading” and “mind reading”. The cognitive error “overgeneralizing” accounted for the most of the variance in depressive symptoms (15 %), while the error “mind reading” for the most of the variance in anxiety symptoms (20 %). However, the cognitive error “selective abstraction” did not appear as a significant predictor of any symptom. This is inconsistent with Schwartz and Maric’s (2015) finding that both overgeneralizing and selective abstraction were predictors of depression. On the other hand, the error “mind reading” and “underestimation of ability to cope” were found to be significant predictors of anxiety symptoms in two studies (Maric et al. 2011; Schwartz and Maric 2015). Some previous studies with the original measure showed that the error “overgeneralizing” was predictive of both anxiety and depressive symptoms (Epkins 1996; Weems et al. 2001), while the error “personalization” was predictive of manifest anxiety (Weems et al. 2001). Contrary to the hypothesis that “underestimation of ability to cope” is the error specific to anxiety symptoms (Maric et al. 2011), our study showed that it was predictive of depressive symptoms. Similarly, the error “selective abstraction” had previously been identified as a significant predictor of depressive symptoms (Weems et al. 2007), but was not the case with the present study’s results.

Further, we compared the CNCEQ-R cognitive errors among youth with anxiety disorders only, youth with depression only, youth with comorbid anxiety disorders and depression, and youth with externalizing symptoms only. The results indicated that distorted cognitive processing was significantly more apparent among youth with depression only than among youth with an anxiety disorder or externalizing symptoms only. Specifically, significant differences among the groups were observed for “overgeneralizing” and “mind reading”. Thus, the overall distorted cognitive processing and the cognitive errors “overgeneralizing” and “mind reading” might distinguish youth with clinical depression only from youth with anxiety disorders or externalizing disorders only. The present findings are in line with other studies with depressed youths (e.g. Curry et al. 2006; Kempton et al. 1994; Kingery et al. 2009; Ginsburg et al. 2009), supporting the original theory that cognitive errors are substantially apparent in depression (Beck et al. 1979). An important finding in this study is that youth with only anxiety did not report substantially more cognitive errors than youth with externalizing disorders. This is unexpected considering that, despite very high levels of the comorbidity between internalizing and externalizing symptoms (e.g., Bilgiç et al. 2013; Cosgrove et al. 2011), previous studies reported differences in cognitive profiles between the two types of symptoms (Leung and Wong 1998).

Taken together, the findings from both types of analyses showed that cognitive errors are associated with depressive and anxiety symptoms, and that these two types of symptoms might be distinguished by the cognitive errors “overgeneralizing” and “mind reading”. However, cognitive errors are significantly more apparent in clinical depression than in anxiety disorders, mixed anxiety-depression, or externalizing symptoms. There are a few possible explanations for these observations. First, some previous studies with the original CNCEQ version demonstrated that greater severity of internalizing problems is associated with increasing magnitudes of cognitive distortions (e.g., Leitenberg et al. 1986; Leung and Wong 1998; Pereira et al. 2012). Thus, it is possible that the cognitive errors associated with depressive symptoms manifest more clearly compared to those with anxiety, especially when depressive symptoms reach clinical levels. Second, there may be additional cognitive processes besides cognitive distortions that operate among children and adolescents expressing clinically significant depressive symptoms then anxiety disorders. For example, attributional style and cognitive errors mediated the relationship between competence and depression, as well as moderated the relationship between negative life events and depression (Cole and Turner 1993). In a study with treatment for adolescents with depression, cognitive errors moderated depression levels over time (Curry et al. 2006). On the other hand, judgment biases demonstrated incremental validity over interpretation biases (i.e., the CNCEQ) in predicting diagnostic status in anxiety disorders (Cannon and Weems 2010).

The above findings need to be considered in light of some limitations. First, although the response rate was above 80 %, possible biases may still result from non-participation, for example those with more overt symptoms may have greater tendency to not participate. Second, this study relied on self-report of internalizing symptoms and did not include scales for evaluating externalizing symptoms. Third, the present sample of children aged up to 12 years was relatively low, and it would be important to replicate the study across bigger samples. Fourth, the present study used one scale for anxiety and depressive symptoms and it would be relevant to consider using different measurement scales to allow investigation into how cognitive errors relate to different aspects of anxiety, such as trait, manifest, and anxiety sensitivity. Fifth, we found it useful to combine the data from the community with the clinical sample due to its wide heterogeneity in diagnosis. Future studies may seek to investigate the role of cognitive errors on a single diagnosis in order to yield greater clinical relevance for that type of psychopathology. Finally, the cross-sectional design of this study did not allow for the estimation of true causal relationships between the cognitive errors and symptoms. Therefore, a longitudinal design study using the CNCEQ-R is needed in order to further evaluate the role of cognitive errors in the development, maintenance, and exacerbation of different symptoms among children and adolescents. Of particular importance would be to incorporate and compare different cognitive models, such as examining the role of dysfunctional schemas and automatic negative thoughts together with cognitive errors. Such an investigation has the potential to identify new distinguishing characteristics among different types of psychopathology.