Introduction

To date, two different approaches have been widely used to describe psychopathology in children and adolescents: the clinical-diagnostic approach and the empirical-quantitative approach. These two approaches have provided competing models for research and clinical practice on child psychopathology for some time, but today the focus is moving towards the possibility of integrating the two approaches rather than privileging one or the other [5].

The first approach is mainly based on the diagnostic and statistical manual of mental disorders (DSM) [8], an expert-based psychiatric categorical system, characterized by a “top down” approach. In this categorical system, disorders reflect current clinical practice and are defined by rather arbitrary sets of criteria. Diagnoses are made by means of semi-structured or structured interviews, designed to operationalize the criteria of the DSM nosological constructs mainly by asking questions whose answers are coded as indicating whether criteria for diagnoses are met or not. These methods require trained staff and are time consuming and expensive. The empirical-quantitative approach, on the other hand, is based on psychometric concepts and the use of quantitative procedures to empirically determine which characteristics tend to co-occur and to constitute ‘syndromes’ in large samples of children or adolescents. In this approach, assessment proceeds in a ‘bottom up’ way, by obtaining scores for specific descriptors of children’s functioning. Scores are subsequently aggregated into scales for measuring psychopathology and other aspects of functioning. Each child’s scale scores can be compared with scores for normative samples, to evaluate their degree of deviance [5]. The Achenbach system of empirically based assessment (ASEBA) is the main representative of the empirical–quantitative approaches. Generally, the following questionnaires are used to gather ASEBA information: child behaviour checklist (CBCL), youth self-report (YSR), and teacher’s report form (TRF). Eight empirically derived syndrome scales (syndrome scales) were built by exploratory and confirmatory factor analysis [4]. These questionnaires are easy to use and have proven to be efficient and low-cost measures for identification of behavioural and emotional problems in children and adolescents, although they do not provide psychiatric diagnoses. The CBCL, as with other rating scales, can be useful for such clinical purposes as screening, treatment planning, progress monitoring, and outcome assessment [44].

In an attempt to overcome this limitation, Achenbach, Dumenici, and Rescorla [4] developed a new scoring system based on consensus between clinicians that allows better correspondence between the vastly adopted CBCL scales and the currently employed DSM-IV diagnostic criteria. The resulting six different DSM-oriented scales (DOSs) include affective, anxiety, somatic, attention-deficit/hyperactivity, oppositional-defiant, and conduct problems. These scales allow a continuous measurement and a “top down” approach. Although research supporting the psychometric properties of these CBCL DOSs is growing, less is known about the clinical utility of such scales with respect to their correspondent diagnoses.

Several studies have examined associations between the clinical-diagnostic approach and the empirical-quantitative approach evaluating association between the ASEBA questionnaire score and the presence of DSM diagnoses. Older studies analysed the convergence of diagnoses and scores on the CBCL syndrome scales or the total problems scale. These studies generally show a better prediction of the externalizing problems scale on the corresponding diagnoses, compared with the internalizing problems scale and corresponding diagnoses. Edelbrock and Costello [20] analysed relationships between DSM-III diagnoses based on the diagnostic interview schedule for children (DISC) [34], and the pre-1991 CBCL scales [1] that preceded the 1991 version [2]. They found a considerable overlap between the CBCL syndrome scales scores and DISC diagnoses in their sample of 270 clinically referred children aged 6–16 years and, generally, they found that the externalizing narrow-band scales strongly correspond to their DSM-diagnoses counterparts.

Similarly, Biederman et al. [10], and Chen et al. [12] found an association between the attention problem syndrome scale and the diagnosis of ADHD, between the delinquent behaviour syndrome scales and the diagnosis of conduct disorders, and between the anxiety/depression syndrome scale and the diagnoses of anxiety disorders.

Kasius et al. [31] found that the CBCL total problem score (1991 version) appears to be very sensitive in predicting any DSM-III R diagnoses, while the CBCL syndrome scales showed lower sensitivity in predicting specific DSM-III R diagnoses, particularly for internalized disorder.

More recent studies have investigated the association between both the syndrome scales and the DOSs CBCL/YSR scales, but focusing only on certain scales; for instance, Van Lang et al. [42] investigated the extent to which DOSs of YSR reflects symptoms of DSM-IV anxiety disorders and DSM-IV major depressive disorder, assessed with the revised child anxiety and depression scale (RCADS) [12, 13] in a community-based epidemiological sample. In addition, they examined whether the association between the affective and anxiety problems DOSs and the RCADS scales was stronger than the association between the anxious/depressed and withdrawn syndrome scales and the same RCADS scales. Their results showed that the affective problems DOS had a stronger association with symptoms of DSM-IV major depressive disorder than with the withdrawn and anxious/depressed syndrome scales. However, the anxiety problems DOS had a weaker association with symptoms of DSM-IV anxiety disorders compared with the anxious/depressed syndrome scale. It was concluded that the construction of the DOSs enhanced its correspondence with DSM-IV major depressive disorder, but not with DSM-IV anxiety disorders.

Ferdinand [23, 24] also explored the convergence between scores on anxiety and affective DOSs and correspondent diagnoses derived by anxiety disorders interview schedule for children (ADIS-C) [40] in a referred sample of 277 children and adolescents aged 6–18. Results showed that scores on the CBCL and YSR anxiety problems DOS only moderately predicted DSM-IV disorders. However, the CBCL and YSR scores on the affective problems DOS corresponded closely to DSM-IV major depressive disorder and dysthymia.

Aebi et al. [7] examined the attention problem DOS compared with the attention problem syndrome scale in the identification of ADHD participants and found a better association with the attention problem DOS than with the attention problem syndrome scale.

Recently, in a very interesting study Ebesutani et al. [19] examined the correspondence of CBCL rationally derived DOSs and syndrome scales with clinical diagnoses in a clinically referred sample of children and adolescents. The authors took into consideration all the CBCL DOSs and used the receiver operating curve (ROC) methodology and discriminative ANOVAs to examine the concurrent validity and correspondence of the internalizing and externalizing-related CBCL DSM-oriented scales with the related DSM diagnoses, using a clinically referred sample. They also compared all the DOSs with corresponding syndrome scales and reported that DOSs generally do not add any further clinical utility above that already afforded by the syndrome scales, except for the anxiety problems DOS that was the only scale which evidenced significantly greater correspondence with diagnoses above its syndrome scale counterpart. They did not measure the probability of obtaining related or other diagnoses using the clinical cut-off suggested by the authors.

Several recent studies report that in children and adolescents there is a greater frequency of co-occurrence of psychopathological traits which involve not only the same domain (internalizing or externalizing) but can simultaneously engage both internalizing and externalizing problem domains, i.e. depression symptoms in oppositional defiant disorder or conduct disorders and internalizing symptoms [11, 36].

This co-occurrence could be due to different co-morbid pathological disorders or it may be that the same etiological factors are responsible for such composite clinical expression. It is important to test prediction of both “target diagnoses” (i.e. those with the same name, such as the ODD scale predicting an ODD diagnosis) and of other, non-target diagnoses (i.e. those with related symptoms but a different diagnostic label, such as the ODD scale predicting an ADHD diagnosis).

In the present study we further tested the convergence between the CBCL DOSs with many DSM-IV diagnoses derived from the semi-structured interview K-SADS-PL in a sample of 298 clinically referred children and adolescents aged 6–16. We decided to focus on the ability of all the six CBCL DOSs to predict not only target diagnoses but also other diagnoses. By “target diagnosis” we mean the ability of DOSs to predict the diagnosis corresponding to the traits measured by DOSs (i.e. anxiety problems DOS to diagnoses of generalized anxiety disorder or separation anxiety disorder or any anxiety disorder); by “non target” diagnoses we mean the prediction of diagnoses that were not expected a priori to predict (i.e. affective problem DOS to ADHD).

As seen before, analysing the prediction of DOSs, both on target and non-target, is very important as it allows us to evaluate the validity and the specificity of the scale when considering these aspects of co-occurrence in the developmental psychopathology in children and adolescents.

As our aim was to measure the probability of obtaining target or non-target diagnoses from suggested clinical scores on each DOS, we used logistic regression analysis and corresponding odds ratio (OR) as a useful indicator of the strength of the relationship between clinical score in DOSs and the presence of DSM-IV diagnoses. Logistic regression allowed us to consider multiple scales simultaneously, thus addressing the problem of co-occurrence of psychopathological traits and to include gender and age as covariates.

Furthermore, we compared the convergence of CBCL DOSs with the syndrome scales on target diagnoses.

Methods

Sample

The target sample of the study consisted of 360 children and adolescents aged 6–16 who were consecutively referred, for the first time, for behavioural and emotional problems to the Child Psychiatry Unit of ‘Eugenio Medea’ Scientific Institute (LC) between January 2003 and December 2008.

The routine clinical assessment included the administration of the “kiddie schedule for affective disorders and schizophrenia for school age children—present and lifetime version” semi-structured interview (K-SADS-PL) with all parents and children aged 11–16. Furthermore, parents were asked to fill in the CBCL 6–18, and total IQ was routinely assessed with the Wechsler intelligence scale for children-revised (WISC-R).

Subjects were excluded if they had an associated neurologic, genetic, infectious or metabolic disorder, or a seizure disorder, mental retardation (IQ < 70), pervasive developmental disorders, severe hypoacusis/hypovision or severe linguistic comprehension deficit (n = 62).

The final sample included 298 subjects: 74 females (25 %; mean age 10 ± 2.19) and 224 males (75 %; mean age 10.04 ± 2.27).

Socio-demographic and clinical characteristics of the recruited sample are represented on Table 1.

Table 1 Sociodemographic characteristics of sample (n = 298)
Full size table

CBCL information was provided by mothers (93 %) or fathers or parent substitute (7 %).

Socioeconomic status (SES) was coded, according to information provided by parents, on the basis of the Hollingshead [29] nine-step scale for parental occupation. A score (from 1 to 9) was assigned to each job; the highest of the two scores was used when both parents were employed. Scores ranging from 7 to 9 corresponded to the upper status, scores ranging from 4 to 6 corresponded to the middle status and scores ranging from 1 to 3 corresponded to the lower status, while a score of zero was used when information was provided but could not be scored (i.e. housewives—those self-employed with no other information—or retired). SES distribution is reported in Table 1.

All the children and parents were fluent in Italian.

Procedure

The study protocols were approved by the ‘Eugenio Medea’ Scientific Institute Ethical Committee. Parents’ written informed consent was obtained for all participants.

Instruments

Child behaviour checklist (CBCL 6–18).

The CBCL 6–18 [3, 5] is an empirically based checklist of social competence and behavioural problems, filled out by parents of children and adolescents aged 6–18. The Italian versions of the CBCL 6–18 were obtained using an independent back-translation authorized and approved by T. Achenbach [27].

The CBCL 6–18 is especially valuable when used with routine, such as being filled-in on intake in mental health settings, for screening in educational and medical settings, and for evaluations by child and family service workers and forensic specialist [3, 5].

The eight syndrome scales: anxious/depressed, withdrawn/depressed, somatic complaints, social problems, thought problems, attention problems, rule-breaking behaviour, and aggressive behaviour, are based on factor analyses.

In recent years, Achenbach and Rescorla [3, 5] developed a new scoring system based on consensus between clinicians that allows better correspondence between the vastly adopted CBCL scale and the currently employed DSM-IV diagnostic criteria [3, 4].

The six DSM-oriented scales (DOSs) and the diagnoses which they are meant to represent are (a) affective problems (dysthymic and major depressive disorders), (b) anxiety problems (generalized anxiety disorder, separation anxiety disorder, and specific phobia), (c) somatic problems (somatization and somatoform disorders), (d) attention-deficit/hyperactivity problems (primarily hyperactive, primarily inattentive and combined subtypes), (e) oppositional defiant problems (oppositional defiant disorder), and (f) conduct problems (conduct disorders).

Unlike the syndrome scales, the DOSs were not derived by factor analytic methods, but were constructed through agreement in ratings among 22 highly experienced child psychiatrists and psychologists from 16 different cultures. These experts rated each pre-existing CBCL item for the degree to which it was consistent with criteria for a particular DSM-IV diagnostic category. Items were then matched with a given diagnostic category if rated as “very consistent with the DSM category” by at least 14 of the 22 experts [4], and the DOSs were developed for categories matching six or more items.

Among research conducted to date on the CBCL DOSs, Achenbach et al. [4] reported that, compared with the syndrome scales, the CBCL DOSs evidenced a similar degree of internal consistency, test–retest reliability, and cross-informant agreement. The factor structure of the CBCL DOSs was also found to be supported in a community sample [4].

Independent research studies over decades have found CBCL syndromes and scale score distributions to be replicated in many cultural groups [30]. The findings have also indicated that the mean scores on many CBCL problem scales were quite similar for children from different cultural groups.

Three sets of norms [5, 6, 30] have been developed to reflect distributions of CBCL scale scores from different cultures. The norm group which is applied for a specific society is dependent upon whether its mean total problems score falls within one SD of the omnicultural mean, below that range or above that range: group 1 (relatively low scores), group 2 (intermediate scores), and group 3 (relatively high scores). These norms are used to equalize the T scores calculated across the three groups. In this study, we used the T score based on the set of multicultural norms “group 2”, which applies to the normative sample of the Italian population [5, 30].

Kiddie schedule for affective disorders and Schizophrenia for school age children—present and lifetime version (K-SADS-PL).

The K-SADS-PL [32, 33] is a semi-structured diagnostic interview created to assess current and past episodes of psychopathology in children and adolescents according to DSM-III-R and DSM-IV criteria.

The primary diagnoses assessed with the K-SADS-PL include major depression, dysthymia, mania, hypomania, cyclothymia, bipolar disorders, schizoaffective disorders, schizophrenia, schizophreniform disorder, brief reactive psychosis, panic disorder, agoraphobia, separation anxiety disorder, avoidant disorder of childhood and adolescence, simple phobia, social phobia, overanxious disorder, generalized anxiety, obsessive compulsive disorder, attention-deficit hyperactivity disorder, conduct disorders, oppositional defiant disorder, enuresis, encopresis, anorexia nervosa, bulimia, transient tic disorder, Tourette’s disorder, chronic motor or vocal tic disorder, alcohol abuse, substance abuse, post-traumatic stress disorder, and adjustment disorders.

The K-SADS-PL consists of screens and supplemental diagnostic assessments for 20 psychiatric disorders. Each screen includes key symptoms for each disorder. If a subject screens positive for a key symptom, a supplement with the remaining DSM-IV symptom criteria for the specific disorder is administered. To screen positive, a score of “three” (threshold) must be given for at least one past or current clinically significant symptom of the disorder.

The interview was conducted with the child or adolescent and with the parents separately but for this study only the interview with parents and current diagnoses were taken into account.

Over the course of this study, all interviews were administered by clinical psychologists who had at least 2 years of clinical experience in the administration of such interviews. Additional training and supervision were provided by an expert clinical psychologist (M. B).

Statistical analysis

  1. (a)

    In order to describe the sample, the frequencies of the CBCL DOSs, the syndrome scales and the Total Problems scale scores in the clinical range were computed, as was the prevalence of all current DSM -IV diagnosis assessed by K-SADS-PL. In accordance with the ASEBA multicultural manual [3, 5], we considered a score to be in the clinical range when the syndrome scale and DOS results were T > 69, and the total problems scale T > 63.

  2. (b)

    In order to evaluate the strength of association of both CBCL DOS and syndrome scale scores and corresponding DSM-IV diagnoses, we performed two sets of logistic regression analyses (LR): (i) LRs for each diagnosis with >4 % frequency (major depression, separation anxiety, generalized anxiety, oppositional defiant disorder, and attention-deficit hyperactivity disorder) versus cases with no diagnosis; (ii) LRs with the three broad categories of “any anxiety disorder” (panic disorder, simple phobia, social phobia, separation anxiety, generalized anxiety disorder, obsessive compulsive disorder, avoidant disorder, anxiety disorder NOS), “any mood disorder” (major depression, dysthymia), and “any disruptive behaviour disorder” (oppositional defiant disorder and attention-deficit hyperactivity disorder) versus cases with no diagnosis. For each of these sets of analysis, first each CBCL scale was used as a single predictor (0 = not in the clinical range, 1 = in clinical range), along with age and gender; second, all significant scales (DOS or syndrome scales, as appropriate) in the first analysis were entered simultaneously in an LR to see which remained significant when all significant predictors were used in combination.

The DSM-IV diagnoses were entered in the regression analysis as 1 = present and 0 = no diagnosis present. In this way, diagnosed subjects were compared with “controls” who did not fulfil the criteria for any DSM-IV diagnosis (even though they were subjects who were referred and cannot be equated with general population subjects).

The Wald statistic was used to test the significance (p < 0.05) of independent variables, whereas model f tests (df = 3) were used to test the significance (p < 0.05) of full regression models. The number of tests in this study may suggest the need for a correction for chance findings for multiple tests, such as a Bonferroni procedure. However, the probability of chance findings in the logistic regression analyses was minimized by applying significance tests for the full regression models.

We used the odd ratio (OR) as a measure of effect size to describe the strength of association between two dichotomic data values. In addition, to understand the strength of the association better, we used Rosenthal’s classification of OR [38]: small (OR ≅ 1.5), medium (OR ≅ 2.5), large (OR ≅ 4.0), and very large (OR ≅ 10.0) association.

To test for the significance of the difference between ORs, we evaluated whether their 95 % CIs overlapped.

Results

  1. (a)

    The frequencies of DSM-IV diagnoses and CBCL DOSs, syndrome scales and total problem scale, score in the clinical range.

Table 2 provides an overview of frequencies of DSM-IV diagnoses assessed by K-SADS-PL. In our sample, 194 subjects have at least one diagnosis while 104 subjects have no diagnosis. Of the 194 subjects who fulfilled criteria for a DSM-IV diagnosis, 141(72.7 %) had one, 43 (22.2 %) had two, eight (4.1 %) had three, and two (1 %) had four diagnoses. No subject fulfilled criteria for diagnoses of conduct disorders, anorexia nervosa, bulimia, alcohol abuse, substance abuse or post-traumatic stress disorder. In addition, no subject received a diagnosis of somatoform disorders, and because of this we do not have the possibility to compare the somatic problems DOS and conduct problems DOS with respective target diagnosis.

Table 2 Frequencies of DSM-IV diagnoses assessed by K-SADS-PL (n %)
Full size table

Frequencies of DOSs, syndrome, and total problems scales score in the clinical range, are reported in Table 3.

Table 3 Frequencies of DOSs, syndrome and total problems scales score in the clinical range (n %)
Full size table
  1. (b)

    Association between CBCL DOSs, syndrome, and total problem scale scores and DSM-IV diagnoses.

OR of the first logistic regression analysis, computed separately for each DOS or syndrome scale (used as a single predictor) and each DSM-IV diagnosis, are reported in Table 4, with the exception of diagnoses of enuresis and any tic disorder which were not predicted by any CBCL scale. ORs that remained significant after the backward logistic regression analysis (when all other significant scales—DOS or syndrome scales as appropriate—were taken into account as predictors) are also reported in brackets in Table 4.

Table 4 Significant OR (p < 0.05) in the first set of logistic regression analysis computed for each CBCL Scales (used as single predictor) and DSM-IV diagnoses with gender and age as covariate
Full size table

Overall, the strength of association between the DOSs and the target diagnoses, according to Rosenthal’s classification [38], ranged from medium to very large (OR range 3.61–35.05).

The strength of association between the DOSs and no target diagnoses (e.g. oppositional defiant disorder DOS vs. ADHD or vs. generalized anxiety disorder) ranged from medium to very large (OR range 3.18–24.37), too.

The difference between DOSs and syndrome scales in predicting any diagnoses obtained no statistical significance. Nagelkerke R2 concerning the full statistical models are reported in Table 4.

Anxiety problems

When combined with other significant single-scale predictors, DOS anxiety problems significantly predicted two target diagnoses (generalized anxiety disorder, OR = 3.79; 95 % CI 1.43–9.23; any anxiety disorder OR = 4.16; 95 % CI 2.07–8.33) and two non-target diagnoses (major depression, OR = 5.96; 95 % CI 1.77–20.05; any mood disorder, OR = 6.06; 95 % CI 2.23–16.44). On the other hand, when used alone the scale significantly predicted only one target diagnosis (separation anxiety, OR = 3.61; 95 % CI, 1.41–9.19).

The anxious/depressed syndrome scale, when combined with other scales, significantly predicted four target diagnoses (major depression, OR = 10.26; 95 % CI 2.79–37.72; any mood disorder, OR = 5.31; 95 % CI 1.86–15.11; generalized anxiety disorder, OR = 3.71; 95 % CI 1.4–9.84; any anxiety disorder OR = 3.95; 95 % CI 1.91–8.16). On the other hand, only when used alone the anxiety/depressed syndrome scale significantly predicted two non-target diagnoses (oppositional defiant disorder, OR = 4.14; 95 % CI 1.41–12.18; any disruptive behaviour disorder, OR = 2.14; 95 % CI 1.06–4.37).

Affective problems

When used alone, the affective problems DOS significantly predicted two target diagnoses (major depression, OR = 4.73; 95 % CI 1.36–16.44; any mood disorder, OR = 4.25; 95 % CI 1.49–12.09) and four non-target diagnoses (generalized anxiety disorders, OR = 3.65; 95 % CI 1.41–9.44; any anxiety disorder, OR = 3.04; 95 % CI 1.43–6.48; oppositional defiant disorder, OR = 4.61; 95 % CI 1.59–13.31; any disruptive behaviour disorder, OR = 2.07; 95 % CI 1.02–4.23).When combined with other significant single-scale predictors, the affective problems DOS did not significantly predict any diagnosis.

When used as a single predictor, the withdrawn/depressed syndrome scale predicted corresponding target diagnosis (“any mood disorders”, OR = 4.01; 95 % CI 1.27–12.55) and three non-target diagnoses (separation anxiety OR = 4.28; 95 % CI 1.36–13.46; generalized anxiety disorders, OR = 6.14; 95 % CI 2.17–17.36; “any anxiety disorder” OR = 4.99; 95 % CI 2.04–11.61). When combined with other significant single-scale predictors, the withdrawn/depressed syndrome scale did not significantly predict any diagnosis.

Somatic problems

Since in our sample no subjects received a diagnosis of somatoform disorders, it was not possible to analyse the prediction of the somatic problems scale on its target diagnosis.

When combined with other significant single-scale predictors, the somatic problems DOS was able to significantly predict three non-target diagnoses (separation anxiety, OR = 11.77; 95 % CI 3.18–43.42; generalized anxiety disorder OR = 5.12; 95 % CI 1.17–19.06; “any anxiety disorder” OR = 4.79; 95 % CI 1.40–16.34). Association with the diagnosis of separation anxiety disorder was the highest and it is the only association which among the internalizing disorders may be classified as very large.

When combined with other significant single-scale predictors, the somatic complaints syndrome scale predicted two non-target diagnoses (separation anxiety disorder OR = 24.00; 95 % CI 4.69–122.78; “any anxiety disorder”, OR = 9.20; 95 % CI 1.91–44.27). In addition, when it was used as a single predictor this scale predicted three non-target diagnoses (“any mood disorders”, OR = 12.12; 95 % CI 1.93–76.96; generalized anxiety disorder, OR = 14.84; 95 % CI 2.85–77.37; oppositional defiant disorder OR = 9.73; 95 % CI 1.20–78.98).

Attention-deficit/hyperactivity problems

When combined with other significant single-scale predictors, the attention-deficit/hyperactivity problems DOS was able to significantly predict two target diagnoses with a large association (attention-deficit/hyperactivity disorders OR = 4.93; 95 % CI 2.45–9.92; “any disruptive behaviour disorder”, OR = 4.24; 95 % CI 2.12–8.44). When it was used as a single predictor, this DOS also significantly predicted one non-target diagnosis (oppositional defiant disorder OR = 4.95; 95 % CI 1.73–14.10).

When combined with other significant single-scale predictors, the attention problems syndrome scale was able to predict two target diagnoses with a medium association (attention-deficit/hyperactivity disorders, OR = 2.55; 95 % CI 1.25–5.19; “any disruptive behaviour disorder”, OR = 2.49; 95 % CI 1.25–4.95). When attention problems syndrome was used as a single predictor it was also able to predict two non-target diagnoses (generalized anxiety disorder OR = 2.7; 95 % CI 1.68–6.72; oppositional defiant disorder OR = 3.79; 95 % CI 1.34–10.68).

Oppositional defiant problems

When combined with other significant single-scale predictors, the oppositional defiant problems DOS was able to predict its corresponding target diagnosis with a very large association (oppositional defiant disorder OR = 14.08; 95 % CI 3.28–67.97, any disruptive behaviour disorder, OR = 5.40). In addition, when used alone, this scale predicted two non-target diagnoses (attention-deficit/hyperactivity disorders OR = 4.08; 95 % CI 1.36–12.24; generalized anxiety disorder OR = 7.56; 95 % CI 2.08–27.48).

When combined with other significant single-scale predictors, the aggressive behaviour syndrome scale predicted its corresponding target diagnosis with a very large association (oppositional defiant disorder OR = 25.00; 95 % CI 4.64–134.58; “any disruptive behaviour disorder”, OR = 5.21; 95 % CI 1.98–13.71), and three non-target diagnoses (attention-deficit/hyperactivity disorders, OR = 3.91; 95 % CI, 1.43–10.71; generalized anxiety, OR = 4.82; 95 % CI 1.37–16.95; “any anxiety disorder OR = 3.37; 95 % CI 1.18–9.59).

Conduct problems

It was not possible to assess the ability of this scale to predict its target diagnosis of conduct disorder as in our sample no subjects had a diagnosis of such.

When combined with other significant single-scale predictors, the conduct problems DOS was able to significantly predict oppositional defiant disorder (OR = 8.76; 95 % CI 1.59–34.30), and as a single predictor two non-target diagnoses (attention-deficit/hyperactivity disorders OR = 3.18; 95 % CI 1.08–9.32; any disruptive behaviour disorder, OR = 4.24; 95 % CI 1.50–11.90).

When used as a single predictor, the rule-breaking behaviour syndrome scale was able to predict four non-target diagnoses (oppositional defiant OR = 16.42; 95 % CI 4.60–58.67; attention-deficit/hyperactivity disorders OR = 3.4; 95 % CI 1.25–9.17; “any disruptive behaviour disorder”, OR = 3.79; 95 % CI 1.44–10.01; generalized anxiety disorder, OR = 4.38; 95 % CI 1.15–16.63).

Social problems syndrome scale

The social problems syndrome scale was able to predict “any mood disorder” generalized anxiety disorder and oppositional defiant disorder (OR = 6.37; 95 % CI 1.44–28.13; OR = 7.96; 95 % CI 2.35–26.99; OR = 32.89; 95 % CI 4.83–223.69, respectively). When this scale was used as a single predictor, it was also able to predict all other diagnoses.

Thought problems syndrome scale

The thought problems syndrome scale was able to predict generalized anxiety disorder and diagnosis of any anxiety disorder (OR = 3.92; 95 % CI 1.33–11.56; OR = 3.33; 95 % CI 1.41–7.85, respectively) as a single predictor only. In our sample, no subjects had diagnoses involving thought disorders.

Total problem scale

As expected, the total problem scale was able to predict all diagnoses with an OR ranging from 2.68 (separation anxiety) to 8.64 (oppositional defiant disorder).

Discussion

In the present study we tested the convergence between the CBCL DOSs with many DSM-IV diagnoses in a sample of 298 clinically referred children and adolescents. We decided to focus on the ability of all six CBCL DOSs to predict not only target diagnoses but also diagnoses they were not expected a priori to predict, as our aim was to measure the probability of obtaining a “target” or “non-target” diagnosis by clinical scores on each DOS.

Generally, the ability to predict the corresponding diagnoses is only slightly higher for the DOSs and syndrome scales in the externalizing domain (even though not significant). The newly developed DOSs predicted disorders in the same broad area (i.e. internalizing or externalizing) of the target diagnostic group with a large or very large association, but the DOSs sometimes predicted non-target diagnosis with similar association. In fact, major depression and diagnosis of “any mood disorder” are predicted by both affective problems and anxiety problems DOSs with a slightly better association with the anxiety scale (even though not significant).

In a similar way, diagnoses of generalized anxiety disorder and any anxiety disorder are predicted by both the affective problems and anxiety problems DOSs, with a slightly better association (even though not significant) with the anxiety DOS. Furthermore, generalized anxiety disorder is predicted by the somatic problems DOS and oppositional defiant problems DOS. If we analysed the strength of associations of both affective and anxiety DOSs on target diagnoses, we found good values for both, while previous studies showed better values for the affective DOS than for the anxiety DOS [23, 24, 42]. Contrary to the results of former studies [23, 24], wherein other scales are taken into account, our results showed slightly better values (even though not significant) for the anxiety DOS than for the affective DOS in predicting target diagnoses.

A very interesting result concerns the diagnosis of separation anxiety disorder which is strongly predicted by the somatic problems DOS with the highest association (OR = 11.77; 95 % CI 4.69–122.78) when other scales are taken into account. Furthermore, diagnoses of generalized anxiety disorder and “any anxiety disorder” are also predicted as single predictor by the somatic problems, affective, and anxiety problems DOSs. These results are very intriguing from a clinical perspective and suggest that, among Italian children and adolescents participating in the study, clinical scores in the somatic problems DOS are very strong predictors of the diagnosis of separation anxiety disorder. Bearing in mind the limitation discussed above, the generalizability of these findings needs to be tested across more clinical samples from more countries.

To our knowledge, there are no other studies evaluating the ability of the somatic problem DOS to predict anxiety disorders or other diagnoses, even though other studies have reported that somatic complaints were strongly associated with psychopathology in children and adolescents [21]. The same authors stress the importance of paying attention to children and adolescents with persistent complaints of headaches, stomach aches or musculoskeletal pains due to the high risk of their having a psychiatric disorder.

In the same way, the oppositional defiant problems DOS predicts not only oppositional defiant disorder but also generalized anxiety disorder (OR = 14.08 and OR = 7.56, respectively), suggesting that oppositional defiant problems could be a predictor of generalized anxiety disorders, too. Therefore, when using CBCL as a screening instrument, clinicians should bear this point in mind to correctly plan further assessment.

Concerning the externalizing domain, attention-deficit/hyperactivity disorders are predicted by the attention-deficit/hyperactivity problems DOS, and by the oppositional defiant problems, and the conduct problems DOSs with similar association. Oppositional defiant disorder is predicted by the oppositional defiant problems and the conduct problems DOSs with a very large association (OR = 14.08 and OR = 8.76, respectively) and with a large association with the attention-deficit/hyperactivity problems and the affective problems DOSs (OR = 4.95 and OR = 4.61, respectively).

Comparing results derived from the DOSs and syndrome scales, we note that the syndrome scales showed a good ability to predict target diagnoses in the broad internalizing area with stronger association than the DOSs (although never significant), but they showed a lack of specificity, as the same diagnosis is predicted by more scales with the same probability. These data are in agreement with previous studies [31] showing that, generally, scores in the clinical range of the internalizing syndrome scale were indicative of a broad range of DSM disorder, rather than one specific diagnosis.

In the externalizing domain, results show that the association between syndrome scales and target diagnoses are similar to that between the DOSs and target diagnoses. Syndrome scales are less specific for the diagnosis of attention-deficit/hyperactivity disorders, as they also predict generalized anxiety disorders while both attention-deficit/hyperactivity problems DOS and attention problems syndrome scale predicted target diagnoses with medium association. These results confirm those of previous studies, in particular that of Aebi et al. [7].

The social problems and thought problems syndrome scales do not have a corresponding diagnosis. The social problems scale predicted all diagnoses, not surprisingly, as social problems may be common to many diagnoses. The thought problems syndrome scale does not predict any diagnoses independently of other scales.

Limitations

As regards our sample, there are some points that should be evaluated carefully.

Even though our sample is a clinically referred sample, 65 % of the subjects have at least one diagnosis while 35 % of subjects have no diagnosis; nevertheless, the percentage of children with no diagnosis is similar to that reported by other studies with clinically referred samples [23, 24, 31]. Contrarily to previous studies, our sample has a lower comorbidity rate: 27 % of the subjects received two or more diagnoses. Other studies reported that 54–84 % of children and adolescents with ADHD may meet criteria for oppositional defiant disorder (ODD) [15, 22, 35]. In our sample, these percentages are not present which could be due to the fact that they were referred for the first time or due to the use of different diagnostic interviews.

We found very different base rates for the clinical range on the CBCL scales and for some diagnoses, such as affective problems DOS and major depression or any mood disorder. As predictive accuracy is influenced by the difference between the base rate and the selection ratio, it would be desirable to have a very similar base rate and selection ratio. Unfortunately, this is not the case of the affective problems DOS and any mood disorders, where very different base rates might have affected the odds ratio obtained. On the other hand, the main aim of this work was to estimate the predictive accuracy of the CBCL DOS in a clinical sample that could adequately represent a population to which these scales are usually applied.

In addition, as in our sample there were no subjects with a diagnosis of somatoform disorder, the somatic problems DOS could not be evaluated in relation to target diagnoses. As far as the diagnosis of somatoform disorder is concerned, it should be considered that the DSM-IV diagnostic criteria were originally established for adults and that no child-specific alternative system has been developed. Therefore, the current classification lacks a paediatric research base [39]. The results from this study could shed light on the fact that somatic symptoms are often comorbid with depressive symptoms but more specific to separation anxiety disorders. The diagnosis of a somatoform disorder involves a continuum that ranges from everyday aches and pains to disabling “functional symptoms”. In children and adolescents, psychiatric disorders such as depression and anxiety disorder often start with physical complaints, for example poor concentration, fatigue, weight loss, an increase in headaches, stomach aches, and chest pains.

Regarding the diagnosis of conduct disorder, in our sample none of the subjects met the criteria for such. It may be that children with the most severe externalizing problems are not typically referred to our Institute but to specialized centres or that parents and children tend to under-report these kinds of problems in K-SADS interviews.

Furthermore, the fact that the rate of conduct disorder in Italy seems to be lower than in other countries should be considered. Frigerio et al. [28], in the only Italian epidemiological study, reported that the percentage of Italian adolescents with externalizing disorders was only 1.2 %.This percentage is in contrast with findings from other studies [9, 16, 17, 25, 26, 41, 43] which reported prevalence estimates of 4–7 %. In addition, the recent ESEMeD-WMH study in an adult population [18] has indicated that, compared with other European countries, in Italy the prevalence of impulse control (0.3 % 95 % CI 0.1–0.5) and substance use disorders (0.1 % 95 % CI 0–0.2) is the lowest.

Another limitation concerns the tools used in this study. We chose KSADS-PL to assess current and past episodes of psychopathology as the interview was included in routine clinical assessment. This interview is only one among many standardized diagnostic interviews and previous studies [14] which revealed little agreement between diagnoses made from the K-SADS versus the diagnostic interview schedule for children (DISC). Moreover, meta-analyses of kappa coefficients for agreement between diagnoses made from standardized diagnostic interviews versus clinical evaluations have yielded modest agreement for many diagnoses [37]. Thus it is possible that disagreements with K-SADS diagnoses (as with diagnoses based on any other single assessment procedure) at least partly reflect the fallibility of that procedure, rather than disagreements with the “true” diagnoses.

Another limitation is the fact of focus being only on the CBCL parent report without taking into account the youth report. Therefore, an interesting future direction would be to consider the CBCL not only being completed by the parents but also the YRS being completed by the child or adolescent, especially with regard to the analyses of internalizing scales. In fact, several studies reported that internalizing symptoms are more reliably reported by the youths’ than the parents’/teachers’ questionnaire and this could lead to an improvement in the ability of internalizing scales to predict corresponding diagnoses.

In this study, we compared children who received a diagnosis with children without a diagnosis, rather than comparing children who had received a diagnosis of a disorder with children without the same disorder. We decided to use logistic regression rather than ROC analyses because logistic regression allowed us to consider multiple scales simultaneously, thus addressing the problem of co-occurrence of psychopathological traits, and to include gender and age as covariates. In this way, although our results may have been less robust, they meet our target of disentanglement of co-occurrence of psychopathological traits better, as the focus of our study was on the ability to predict any diagnoses while taking into account the covariation of the other scales.

Conclusion

This study is the first that investigates the ability of all the CBCL DOSs to predict any diagnoses while taking into account co-variation of the other scales. We showed a significant correspondence of the CBCL DOSs with related DSM-IV diagnoses. These associations were generally more specific but weaker (even though not significantly) than the association between the syndrome scales and related DSM-IV diagnoses.

We also found that the somatic problems DOS is a very strong and specific predictor for diagnosis of separation anxiety and that the oppositional defiant problems DOS is not only a predictor of the oppositional defiant disorder but also a strong predictor of generalized anxiety disorder. These latter results suggest that when using CBCL as a screening instrument in a clinical setting, the clinician should bear in mind that they are not specific for their target diagnosis. Use of the CBCL DOSs can really combine the “top down” approach identifying diagnoses which are DSM-oriented with a continuous measurement, by comparing each child’s scales scores with normative samples to evaluate the degree of its deviance from normality.