Introduction

Anxiety disorders are among the most common psychiatric disorders in childhood and adolescence, with an estimated worldwide prevalence of 6.5% (95% confidence interval [CI]: 4.7–9.1%; Polanczyk et al. 2015). Affected children may experience functional impairment, including, e.g., difficulties in social functioning (Kingery et al. 2010) and impairment at school or during leisure time (Essau et al. 2000). Moreover, childhood anxiety disorders predict the presence of a range of psychiatric disorders in adolescence (Bittner et al. 2007). Thus, early diagnosis and treatment are important.

In the last decades, research efforts have focused on the assessment of anxiety disorders in children and adolescents. Besides diagnostic interviews, rating scales constitute an important assessment approach. In the international literature, a number of different anxiety questionnaires have been developed, some of them corresponding with diagnoses from the Diagnostic and Statistical Manual of Mental Disorders (DSM, currently 5th edition, American Psychiatric Association 2013) or the International Classification of Diseases (ICD, currently 10th edition, World Health Organisation 1993). DSM-based scales include for example the Screen for Child Anxiety Related Emotional Disorders (SCARED; e.g., Birmaher et al. 1997, 1999; Muris et al. 1999b), the Spence Children’s Anxiety Scale (SCAS; Spence 1998), and the Revised Child Anxiety and Depression Scale (RCADS; Chorpita et al. 2000). An instrument which has been especially designed for the assessment of anxiety disorders according to the most current version of the DSM, the DSM-5, is the Youth Anxiety Measure for DSM-5 (YAM-5; Muris et al. 2017). In accordance with the common recommendation to involve different informants in the diagnostic process (Achenbach et al. 2008; Comer and Kendall 2004), these scales are all available as self-report and parent versions (e.g., Birmaher et al. 1997, 1999; Ebesutani et al. 2010, 2011; Muris et al. 2004, 2017; Nauta et al. 2004). They have been shown to have good psychometric properties for different cultures and for community samples as well as for clinical samples (for a meta-analysis of the psychometric properties of the SCARED see Hale et al. 2011; for a systematic review of the factor structure and reliability of the SCAS see Orgilés et al. 2016; parent version of the SCARED: e.g., Muris et al. 2004; parent version of the SCAS: e.g., Nauta et al. 2004; RCADS self-report or parent version: e.g., Bouvard et al. 2015; Chorpita et al. 2000, 2005; Ebesutani et al. 2010, 2011; Koesters et al. 2015; Park et al. 2016; YAM-5 self-report and parent version: Muris et al. 2017). However, only a few studies have explored the diagnostic value of teacher ratings in the assessment of anxiety disorders (e.g., Epkins 1993, 1996).

In Germany, clinicians and researchers often use the ICD-10 and DSM-IV-based Self-Report Rating Scale for Anxiety Disorders (SRS-AD; German: Selbstbeurteilungsbogen fuer Angst- und Zwangsstoerungen; Doepfner et al. 2008) and the Parent/Teacher Report Rating Scale for Anxiety Disorders (PRS-AD/TRS-AD; German: Fremdbeurteilungsbogen fuer Angst- und Zwangsstoerungen; Doepfner et al. 2008) for assessing the most common anxiety disorders in children and adolescents. The rating scales cover symptom criteria of separation anxiety disorder, generalized anxiety disorder, social phobia, and specific phobias. Since the criteria for these anxiety disorders did not substantially change from DSM-IV to DSM-5, the instruments also correspond with the respective DSM-5 diagnostic criteria. Two additional items serve to screen for the presence of an obsessive-compulsive disorder. However, these are not subject to this article as they were not part of a former version of the instrument and as obsessive compulsive disorders have been moved to another section in the DSM-5. Moreover, it is important to note that the rating scales do not provide any information about the presence of a panic disorder or selective mutism, which are also described in the DSM-5 section on anxiety disorders.

The psychometric properties of the SRS-AD and PRS-AD have already been analysed using a community sample of children or adolescents and their parents (Doepfner et al. 2008), as well as in a smaller clinical sample of children and adolescents with anxiety disorders and their parents (Dose et al. 2015), which formed part of the sample used in the present study. In these analyses, most of the subscales and the overall anxiety scale of both the SRS-AD and the PRS-AD showed satisfactory internal consistency (Doepfner et al. 2008; Dose et al. 2015). Also, exploratory factor analyses on the items of the SRS-AD and the PRS-AD revealed factors largely corresponding with DSM or ICD diagnoses of anxiety disorders (Doepfner et al. 2008; Dose et al. 2015). Differing results were found for some items of the SRS-AD belonging to the two subscales Separation Anxiety Disorder and Specific Phobias (defined a priori) when analysed in the clinical sample of children and adolescents with anxiety disorders (Dose et al. 2015). Regarding the relation between the child/adolescent and parent ratings, analyses in the community sample found a significant difference in the mean ratings of parents and children; the children and adolescents rated themselves as more anxious than their parents did (Doepfner et al. 2008). However, in the smaller clinical sample of children with anxiety disorders, no significant difference in mean ratings was found (Dose et al. 2015). In both the community sample and the smaller clinical sample, correlations between parent and child ratings were moderate (Doepfner et al. 2008; Dose et al. 2015). Teacher ratings on the TRS-AD have not been analysed, yet.

In the current study, we examined the psychometric properties of the SRS-AD in a clinical sample of children or adolescents with psychiatric disorders, those of the Anxiety Rating Scale for parents (PRS-AD) and those of a shortened Anxiety Rating Scale version for teachers (TRS-AD). In accordance with previous research on the SRS-AD and the PRS-AD, we expected to find satisfactory internal consistencies for most of the subscales and the total scale of all questionnaires. As the examination of the factor structure of the SRS-AD and PRS-AD has only been exploratory to date, we performed confirmatory factor analyses to compare alternative factor models separately for the child, parent and teacher samples. Given the overlap between diagnostic criteria for different anxiety disorders and the high rate of comorbidity among these disorders (Essau et al. 2000), we expected a second-order model with first-order factors according to the different subscales and a second-order factor that explains the correlations between the first-order factors to provide satisfactory fit for the empirical data. Moreover, we expected the ratings of children, parents and teachers on the SRS-AD, PRS-AD and TRS-AD to discriminate between children with anxiety and non-anxiety disorders. Also, we hypothesised to find low-to-moderate correlations between the ratings of children/adolescents, parents and teachers for the SRS-AD, PRS-AD and TRS-AD as this would be consistent with previous research on the questionnaires and with the frequent finding that parent–child, teacher–child and parent–teacher agreement in the assessment of psychiatric symptoms is generally only low-to-moderate (e.g., Achenbach et al. 2008; Doepfner et al. 2014). Finally, we examined age and gender differences on the subscales of the three questionnaires.

Method

Participants

The clinical sample included children and adolescents referred to outpatient units of the Department of Child and Adolescent Psychiatry and Psychotherapy and the School for Child and Adolescent Cognitive Behaviour Therapy (AKiP) at the University Hospital of Cologne, Germany, between November 2000 and October 2012, their parents and teachers. Patients, their parents and teachers complete the SRS-AD and the PRS-AD/TRS-AD as part of the usual intake assessment. Until 2008, an earlier version of the instruments was used, which included the same items for the assessment of anxiety disorders, but not the items covering obsessive-compulsive disorders or competences. In the aforementioned period, the SRS-AD and PRS-AD/TRS-AD were completed for a total of N = 881 patients (587 self-ratings, 827 parent ratings and 448 teacher ratings). For each questionnaire, cases with more than 10% missing values were excluded from the analysis, resulting in a total sample of N = 879 children and adolescents and the following sub-samples: the SRS-AD was completed by n = 585 children, adolescents and young adults aged between 6 and 21 years (M = 14.09, SD = 2.52; n = 295 [50%] female). Parents answered the PRS-AD for n = 821 patients ranging in age from 3 to 21 years (M = 12.04, SD = 3.60; n = 402 [49%] female). Teachers or caregivers completed the TRS-AD for n = 378 students (age range = 4–19 years, M = 11.22, SD = 3.42; n = 185 [49%] female). Measures from all three informants were available for a total of n = 203 children and adolescents.

Table 1 shows the main diagnoses for the children, adolescents and young adults made during the intake assessment by a psychologist with the aid of a semi-structured clinical interview with a diagnostic checklist covering ICD-10 and DSM-IV diagnoses (Diagnostic Checklist for the Assessment of Anxiety Disorders, German: Diagnose-Checkliste fuer Angststoerungen; Doepfner et al. 2008). About 37% of the youths for whom the SRS-AD was available, about 39% of the youths for whom the PRS-AD was completed and about 40% of the youths for whom the TRS-AD was answered had the primary diagnosis of an anxiety disorder, i.e., a primary diagnosis of an agoraphobia with or without panic disorder, a panic disorder, social phobia/social anxiety disorder of childhood, specific phobia/phobic anxiety disorder of childhood, generalized anxiety disorder, separation anxiety disorder, mixed anxiety and depressive disorder, other mixed anxiety disorder, or unspecified anxiety disorder. For the current analyses, we included the aforementioned diagnoses in the anxiety subsamples as we expected youths with these diagnoses to show high values on the rating scales. In the subsample of children, adolescents and young adults with the diagnosis of an anxiety disorder, the SRS-AD was available for 218 participants aged 6 to 20 years (M = 14.25, SD = 2.46; n = 116 [53%] female), the PRS-AD was completed for 319 participants aged 4 to 20 years (M = 12.10, SD = 3.73; n = 170 [53%] female) and the TRS-AD was answered for 151 participants aged 4 to 18 years (M = 11.00, SD = 3.60; n = 83 [55%] female).

Table 1 Children’s and adolescents’ ICD-10 diagnoses
Full size table

Procedure

The Medical Ethical Committee of the University Hospital of Cologne, Germany, declared that no formal consent is required for this type of retrospective research involving data collected in routine clinical care and analysed anonymously. All procedures performed in this study involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. All parents declared their consent that data on their child gathered during the time of treatment at the Department of Child and Adolescent Psychiatry and Psychotherapy and the School for Child and Adolescent Cognitive Behaviour Therapy (AKiP) at the University Hospital of Cologne, e.g. questionnaire data, may be used for research purposes. Adolescents/young adults who were already of legal age provided this consent themselves.

Measures

The SRS-AD and the PRS-AD are rating scales for the assessment of the most common anxiety disorders in childhood and adolescence. The SRS-AD is a self-report questionnaire for children and adolescents, whereas the PRS-AD was originally designed to be answered by parents, other relatives or teachers. Both rating scales consist of 31 items, each covering a specific symptom behaviour (e.g., “is worried that he/she could lose his/her parents or that something bad could happen to his/her parents”). The items can be grouped into four subscales defined a priori as: Separation Anxiety Disorder (SAD, 10 items), Generalized Anxiety Disorder (GAD, 7 items), Social Phobia (SOP, 7 items) and Specific Phobias (SP, 7 items). Two additional items serve to indicate the presence of an obsessive-compulsive disorder and were not part of a former version of the instruments. In addition, there are eight further items for the assessment of competences concerning sociability and confidence. These items serve to identify resources which, besides symptom information, may be important in therapy.

The current analyses of the self- and parent ratings focus on the 31 items for the assessment of anxiety disorders (i.e., the two items for the assessment of obsessive-compulsive disorders were not included). The teachers originally completed the same questionnaire as the parents. However, the analyses of the teacher sample used a shortened version of the PRS-AD, the TRS-AD, as it turned out that the original questionnaire consists of many items covering behaviour that cannot directly be observed by teachers (e.g., sleep problems, nightmares) and thus, there were some items with many missing values. For the teacher version used for the current analyses, items with more than 15% missing values in the original data set were excluded. This procedure led to the exclusion of all items of the Specific Phobias scale, six items belonging to the Separation Anxiety Disorder scale and one item of the Generalized Anxiety Disorder scale. The resulting TRS-AD consists of 17 items that can be grouped into three subscales (Separation Anxiety Disorder, Generalized Anxiety Disorder, and Social Phobia).

The severity of each item is rated on a 4-point Likert scale ranging from 0 (“not at all”) to 3 (“very much”). The overall score is computed by taking the average of the individual ratings across all items. For the present analyses, the overall score is computed from the 31 or 17 items, as appropriate.

Data Analyses

All analyses were conducted using SPSS Version 22 and Mplus (for the confirmatory factor analyses). For the primary analyses, we used all available questionnaires, irrespective of the children’s or adolescents’ diagnoses, to increase the sample size. The percentage of missing values per item was ≤1% in the final subsamples except for item 31 in the patient and parent sample (3% in each sample). Given their low number, missing values were not replaced. For analyses on the scale level (i.e., the analyses to determine discriminating validity and the agreement between different informants), scale scores were computed by averaging the available item scores. As the diagnoses in the different subsamples were very heterogeneous (cf. Table 1), all analyses were rerun in the subsamples including only questionnaires for children, adolescents and young adults with an anxiety disorder.

Confirmatory factor analyses were computed separately for the three subsamples (child, parent and teacher samples). In the child and parent samples, three different models were analysed and compared. First, we examined a unidimensional model assuming an overall anxiety factor that influences all items (model I). Second, we considered a first-order correlated-factors model with four first-order factors according to the a priori defined scale structure (Separation Anxiety Disorder, Generalized Anxiety Disorder, Social Phobia, and Specific Phobias; model II). In this model, each item was specified to load on one factor and the factors were allowed to correlate freely. For identification purposes, the loading of the first item per factor was fixed to one. Third, we analysed a second-order model with four first-order factors and one second-order overall anxiety factor that explains the correlations among the first-order factors (model III; see Reise et al. 2010). To identify this model, the loading of the first item per first-order factor and one loading in the second-order factor were fixed to one.

In the teacher sample, we compared a unidimensional model (model I), a first-order model implying three factors according to the a priori expected scale structure (Separation Anxiety Disorder, Generalized Anxiety Disorder, and Social Phobia; model II) and a second-order model (model III).

As a 4-point Likert scale was used, we considered the item scores as ordered categorical data and employed the robust weighted least squares with mean and variance adjustment estimator (WLSMV) for model estimation. This estimator uses polychoric correlations (Brown 2006; Muthén and Muthén 1998–). Missing data were dealt with by using pairwise present analysis, which is the default procedure in Mplus (Muthén and Muthén 1998–).

Several different indices were considered to evaluate model fit. Since models are often rejected in large samples when only chi-square statistics are considered (Schermelleh-Engel et al. 2003), we additionally evaluated the chi-square value (χ²) relative to the degrees of freedom (df). Lower χ2/df-ratios indicate a better model fit; a χ2/df-value of 2 indicates “good” model fit and a value of 3 indicates “acceptable” model fit (Schermelleh-Engel et al. 2003). We also used the comparative fit index (CFI) and the root mean square error of approximation (RMSEA) as indices of goodness of fit. A CFI above 0.90 and an RMSEA below 0.08 indicate acceptable model fit (Browne and Cudeck 1993). Following Kline (1994), factor loadings of λ > .30 were considered as acceptable.

Nested models were compared using the chi-square difference test provided by Mplus (Muthén and Muthén 1998–2015). The unidimensional model and the second-order model are nested in the less restricted first-order model (see Reise 2012). When the chi-square difference test provides a significant result, the null hypothesis of equal model fit of the compared models is rejected and the less restricted model should be retained. When the test provides a non-significant result, the more restricted model does not fit the data significantly worse than the other model and, as a result, should be favoured (Schermelleh-Engel et al. 2003).

Internal consistencies of the subscales and total scales were determined by computing Cronbach’s alpha separately for the child, parent and teacher samples. A satisfactory test or scale required a minimum internal consistency of α = .70 (Nunnally 1978). In addition, part–whole corrected item–scale correlations (rit) were determined for the total score and different subscale scores. Item–scale correlations were considered to be moderate if .30 ≤ rit ≤ .50 and high if rit > .50 (Bortz and Doering 2006).

Discriminating validity was assessed by comparing the ratings for children with any anxiety disorder to children with a non-anxiety disorder using independent-samples t-tests. The Bonferroni–Holm procedure was used to control for the familywise error rate in analyses involving subscales of the same questionnaire (Holm 1979). Cohen’s d was employed as a measure of effect size (Cohen 1988).

To examine if there are significant differences between child, parent and teacher ratings, matched-pairs t-tests were computed. Again, the Bonferroni–Holm procedure was applied and Cohen’s d was considered as a measure of effect size. Parent–child, parent–teacher and child–teacher agreements were examined using Pearson’s correlation coefficients between the corresponding PRS-AD, TRS-AD and SRS-AD subscales.

To analyse the relationship between the questionnaires’ subscales and total scale and the age of the children and adolescents, Pearson’s correlation coefficients were calculated. Independent-samples t-tests were performed to examine gender differences on the subscales and total scale of the three rating scales. Again, we adopted the Bonferroni-Holm procedure (Holm 1979) to control for the familywise error rate in analyses using subscales of the same questionnaire and used Cohen’s d as a measure of effect size (Cohen 1988).

Results

Factor Structure

In the parent sample including participants with anxiety and non-anxiety disorders, the first-order model with four correlated factors (model II) fitted the data significantly better than the unidimensional model of the PRS-AD (model I; Δχ2 = 882.67, df = 6, p < .01). The RMSEA indicated an acceptable fit of the first-order model, while the CFI fell in the borderline range and the χ2/df-ratio was not satisfactory (see Table 2). In this model, all standardized factor loadings were significant and ranged from .46 to .93 (see Table 3). The model-implied correlations between the first-order factors were mostly moderate and ranged from .33 to .56. This suggests that the factors share a certain amount of common variance, making it useful to test a second-order model which may explain the correlations between the first-order factors. The second-order model (model III) provided similar fit indices as the first-order model. Again, the RMSEA was acceptable, the CFI fell in the borderline range and the χ2/df-ratio was unsatisfactory. The loadings of the first-order factors on the second-order factor were all significant and ranged from .58 to .77. The chi-square difference test indicated that this model provided a significantly worse data fit than model II (Δχ2 = 18.61, df = 2, p < .01). However, according to Brunner et al. (2012), first-order and second-order models might be compared further by examining the residual correlations, which are computed as the difference between the model-implied correlations between the first-order factors in the second-order model and the corresponding correlations in the first-order model. In our analyses, these residual correlations ranged from −.08 to .06. Referring to Brunner et al. (2012), these low residual correlations support the second-order model although the chi-square test was significant.

Table 2 Confirmatory factor analyses comparing different models for the SRS-AD, PRS-AD and TRS-AD in samples of children, parents and teachers
Full size table
Table 3 Confirmatory factor analyses: Factor loadings for the first-order (non-modified) solutions
Full size table

Based on the modification indices and content-related reflections, we made some modifications to the second-order model to increase the model fit. The residuals of items 1 and 4, items 2 and 3 and items 5 and 6 were allowed to correlate freely as the analysis had produced high modification indices for these model specifications and because all of these items belong to the Separation Anxiety Disorder scale and cover sleep-related anxieties, suggesting these modifications are textually plausible. In addition, item 11 (“is afraid in many different situations”) was allowed to additionally load on the Separation Anxiety Disorder, Social Phobia and Specific Phobias factors. Besides the high modification indices for this modification, this makes practical sense because this item may cover behaviour relevant to all anxiety disorders and might not be specific enough to belong to the Generalized Anxiety Disorder factor alone. Item 17 (“has sleep problems”) was allowed to load on the Separation Anxiety Disorder factor on which other sleep-related items also load. Item 18 (“is afraid of failing”) was allowed to additionally load on the Generalized Anxiety Disorder factor as it covers behaviour not only relevant to social phobia, but also to generalized anxieties. For the resulting model, the fit indices revealed a good fit for the data (see Table 2).

As for the PRS-AD rated by parents, the first-order model (model II) fitted the data for the SRS-AD significantly better than the unidimensional model (model I; Δχ2 = 430.72, df = 6, p < .01). The CFI, the RMSEA and the χ2/df-value indicated a satisfactory fit of this model (see Table 2). The factor loadings in this model were significant and ranged from .38 to .81 (see Table 3). The model-implied correlations between the factors ranged from .44 to .67. The chi-square difference test indicated that the second-order model (model III) fitted the data significantly worse than the first-order model (Δχ2 = 20.84, df = 2, p < .01). Nevertheless, again we found only low residual correlations (range from −.06 to .11). The loadings of the first-order factors on the second-order factor ranged from .66 to .85. When we applied the same modifications as for the parent sample to the second-order model, model fit still increased (see Table 2).

For the TRS-AD rated by teachers, the first-order three-factor model showed significantly better fit for the data than the unidimensional model (Δχ2 = 227.18, df = 3, p < .01). However, only the CFI indicated an acceptable fit of this model; the RMSEA and the χ2/df-value were not satisfactory (see Table 2). All factor loadings were significant and ranged from .47 to .95 (see Table 3). The correlations between the factors ranged from .43 to .58. Since a second-order solution with three first-order factors is just-identified, the analysis reveals the same goodness of fit for this model as for the first-order solution (Brown 2006). However, it may be useful to test this model to examine the higher-order factor loadings (Brown 2006). In our case, the loadings of the first-order factors on the second-order factor were rather high and ranged from .59 to .80. When we made modifications similar to those made for the parent and child questionnaires, a satisfactory model fit was found (see Table 2). Modifications were as follows: the residuals of items 1 and 4 as well as those of items 2 and 3 were allowed to correlate; item 11 was allowed to cross-load on the Separation Anxiety Disorder and Social Phobia factors and item 18 was allowed to cross-load on the Generalized Anxiety Disorder factor.

When only questionnaires for anxious children, adolescents and young adults were used for the analyses, the proposed models generally showed a reasonable model fit, even if no modifications were made. For both the PRS-AD and the SRS-AD, the first-order solution (PRS-AD: χ2/df = 2.27, CFI = .91, RMSEA = .06; SRS-AD: χ2/df = 1.60, CFI = .92, RMSEA = .05) as well as the second-order solution (PRS-AD: χ2/df = 2.26, CFI = .91, RMSEA = .06; SRS-AD: χ2/df = 1.66, CFI = .91, RMSEA = .06) provided a satisfactory fit for the data. All standardized factor loadings in the first-order models were significant and satisfactory (λ > .30) and the model-implied correlations between the factors were mostly moderate (PRS-AD: r = .22–.46, SRS-AD: r = .32–.70). Moreover, the first-order factors showed significant and substantial loadings on the second-order factors in the second-order model (PRS-AD: λ = .50–.73; SRS-AD: λ = .54–.89). For both rating scales, the chi-square test indicated a significantly worse data fit of the second-order model compared to the first-order model (PRS-AD: Δχ2 = 8.39, df = 2, p = .02; SRS-AD: Δχ2 = 18.11, df = 2, p < .01). For the PRS-AD, the residual correlations (range .01–.08) supported the second-order model. However, for the SRS-AD, this was not completely the case (range of residual correlations .03–.14). For the TRS-AD, the first-order model also provided a reasonable data fit in the sample of anxious children, adolescents and young adults (χ2/df = 2.18, CFI = .95, RMSEA = .09). All standardized factor loadings on the first-order factors were significant and satisfactory; the model-implied correlations between the first-order factors were moderate (r = .39–.65). In the second-order model, the loadings of the first-order factors on the second-order factor ranged from .54 to .89.

Reliability

Cronbach’s alpha coefficient was .88 for the PRS-AD overall anxiety scale rated by parents and .86 for the shortened TRS-AD overall anxiety scale rated by teachers. In both the parent and teacher samples, internal consistencies of the subscales met the criterion of α ≥ .70 (parent sample: α = .70–.85, teacher sample: α = .77–.86; see Table 4). For the SRS-AD rated by children/adolescents, the Cronbach’s alpha coefficient was .89 for the overall anxiety scale and the internal consistency was satisfactory for all subscales except the Specific Phobias subscale (α = .65–.83; see Table 4).

Table 4 Descriptive statistics, internal consistency and range of part–whole corrected item–subscale correlations for the SRS-AD, PRS-AD and TRS-AD rated by children/adolescents, parents, and teachers
Full size table

Most of the part–whole corrected item–subscale correlations were moderate-to-high (rit ≥ 30; see Table 4). Exceptions were found for item 31 (“is afraid of other situations”) of the Specific Phobias subscale rated by parents as well as for items 25 (“I am afraid of animals”) and 31 (“I am afraid of other situations”) of the Specific Phobias subscale rated by children and adolescents. Deletion of single items with a low item–subscale correlation did not lead to a meaningful increase of internal consistency.

When only questionnaires for children, adolescents and young adults with the main diagnosis of an anxiety disorder were considered for the analyses, similar results emerged. In these analyses, the internal consistency was satisfactory for all total scales and subscales but the Specific Phobias subscale in the parent and children/adolescents sample (parent sample: α = .88 for the total scale and α = .67–.86 for the subscales, teacher sample: α = .87 for the total scale and α = .76–.86 for the subscales, children/adolescents sample: α = .89 for the total scale and .58–.83 for the subscales). The part-whole corrected item–subscale correlations were rit ≥ .30 except for item 25 in the parent sample and for items 6 (“get up at night”; Separation Anxiety Disorder subscale), 30 (“I am afraid of crowds”; Specific Phobias subscale) and 31 in the children/adolescents sample.

Discriminating Validity

Independent-samples t-tests revealed significant differences between the ratings for anxious and non-anxious children on the total scale and all subscales of the PRS-AD rated by parents and on the total scale and Social Phobia subscale of the SRS-AD, when Bonferroni–Holm-corrected significance levels were applied. The anxious group had higher mean scores on these scales except for the Generalized Anxiety subscale rated by parents, on which the non-anxious group showed higher scores. The differences on the Separation Anxiety Disorder and Specific Phobias subscales of the SRS-AD just missed statistical significance. No significant difference was observed between anxious and non-anxious children on the total scale score of the TRS-AD rated by teachers. For this instrument, significant group differences emerged on the Separation Anxiety Disorder and Social Phobia subscales (see Table 5). Where effects were significant, effect sizes were small (see Table 5).

Table 5 Discriminating validity: Independent-samples t-tests comparing anxious and non-anxious children on the scales of the SRS-AD, PRS-AD and TRS-AD
Full size table

Parent–Child, Parent–Teacher and Child–Teacher Agreement

When all available questionnaires were included into the analyses, matched-pairs t-tests revealed significant differences between parents’ and teachers’ ratings on all scales (Separation Anxiety Disorder: t = 9.76, d = .51; Generalized Anxiety Disorder: t = 6.63, d = .35; Social Phobia: t = 6.82, d = .36; overall anxiety scale: t = 5.88, d = .31; for all scales n = 359, df = 358, p < .01), between teachers’ and child self-ratings on all scales (Separation Anxiety Disorder: t = −8.35, d = .57; Generalized Anxiety Disorder: t = −3.74, d = .26; Social Phobia: t = −2.28, d = .16; overall anxiety scale: t = −3.05, d = .21; for all scales n = 214, df = 213, p < .05), and between parents’ and child self-ratings on the Separation Anxiety Disorder, the Social Phobia and the Specific Phobias scales (Separation Anxiety Disorder: t = −3.57, d = .15; Social Phobia: t = 3.59, d = .16; Specific Phobias: t = −2.60, d = .11; for these scales n = 535, df = 534, p < .05; Generalized Anxiety Disorder: t = −0.07, df = 534, p = .94; overall anxiety scale: t = −0.75, df = 534, p = .45). Children/adolescents and their parents generally reported more severe symptoms than teachers. Children rated their symptoms on the Separation Anxiety Disorder and Specific Phobias subscales as more severe than their parents, whereas parents thought the children experienced more severe social phobia than children did themselves. Correlations between the ratings from children/adolescents and their parents on the different corresponding subscales covering anxiety symptoms ranged from r = .39 to r = .55. For the agreement between parents anbetween child, parent and teacher ratings on the correspondingd teachers, correlations from r = .21 to r = .42 emerged. The lowest informant agreement was found for the child–teacher pairs: correlations between the corresponding subscales ranged from r = .10 to r = .27 (see Table 6).

Table 6 Informant agreement: correlations between child, parent and teacher ratings on the corresponding SRS-AD, PRS-AD and TRS-AD subscales (Pearson’s correlation coefficients)
Full size table

Considering only the subsample of children, adolescents and young adults for whom all three questionnaires were available (n = 203), matched-pairs t-tests resulted in significant differences between parent and self-ratings on the Separation Anxiety Disorder and Social Phobia subscales, between parent and teacher ratings on all scales and between teacher and self-ratings on all scales. Again, children/adolescents and parents reported more severe symptoms than teachers, the self-ratings were higher than the parent ratings for symptoms of separation anxiety disorder and the parent ratings were higher than the self-ratings on symptoms of social phobia. Correlations between ratings from different perspectives were low-to-moderate and ranged from .43 to .59 for the agreement between parents and children, adolescents and young adults themselves, from .16 to .47 for the parent-teacher agreement and from .07 to .27 for the self-teacher agreement (see Table 6).

When only ratings for children, adolescents and young adults with a diagnosis of an anxiety disorder were considered for the analyses, the results were similar for the analyses involving all available pairs of ratings, except that the difference between the self- and parent ratings on the Specific Phobias subscale was non-significant. The correlations between different rating perspectives were also mostly low-to-moderate in this sample. When only children, adolescents and young adults who had the main diagnosis of an anxiety disorder and for whom all questionnaires were available were included into the analyses (n = 78), significant differences between self- and teacher ratings on all scales, between teacher and parent ratings on all scales and between self- and parent ratings on the Social Phobia scale emerged. Again, correlations were mostly low-to-moderate and highest for the self- and parent ratings.

Age and Gender Differences

For all samples, only low correlations between the children’s/adolescents’ age and the scale scores of the different questionnaires were found (PRS-AD total scale: r = −.08, PRS-AD subscales r = −.32–.12; SRS-AD total scale: r = .08, SRS-AD subscales: r = −.14–.23; TRS-AD total scale: r = −.03, TRS-AD subscales: r = −.20–.05). The results were similar when only participants with a diagnosis of an anxiety disorder were considered. Independent-samples t-tests demonstrated significant gender differences on the total scale (t = –2.99, df = 819, p < .01, d = .21), the Social Phobia subscale (t = −2.54, df = 819, p = .01, d = .17), the Specific Phobias subscale (t = −3.77, df = 819, p < .01, d = .27) of the PRS-AD and on all scales of the SRS-AD (overall anxiety scale: t = −5.28, d = .43; Separation Anxiety Disorder: t = −2.02, d = .17; Generalized Anxiety Disorder: t = −5.25, d = .43; Social Phobia: t = −2.62, d = .22; Specific Phobias: t = −6.47, d = .53; for all scales df = 583 and p < .05), with higher scores found for females. For the TRS-AD, no significant differences were found. The results for the SRS-AD remained the same when we only included participants with a diagnosis of an anxiety disorder into the analyses. However, for both the PRS-AD and the TRS-AD no gender differences were found in this case.

Discussion

The present study examined the psychometric properties of German anxiety questionnaires, the PRS-AD, the TRS-AD and the SRS-AD, in a clinical sample of children, adolescents and young adults (aged 6–21 years), their parents and teachers. It adds to the literature in that it analysed teacher ratings on the German Rating Scale for Anxiety Disorders, performed confirmatory factor analyses on the factor structure and examined the discriminating validity of the different questionnaires. Moreover, most previous studies on DSM-based instruments have analysed large community samples (e.g., Chorpita et al. 2000; Crocetti et al. 2009; DeSousa et al. 2014; Ebesutani et al. 2010, 2011; Essau et al. 2002, 2011a, 2011b; Muris et al. 1999a; Spence 1998; Spence et al. 2003) and/or comparatively smaller clinical samples (e.g., Arendt et al. 2014; Muris et al. 2004; Muris and Steernemann 2001; Park et al. 2016; Weitkamp et al. 2010; Whiteside and Brown 2008). In contrast, we have analysed a relatively large clinical sample. Analyses of clinical samples are important because the rating scales claim to assess disturbances and are mainly used in children and their caregivers who seek treatment. Analyses of community samples, on the other hand, primarily assess variations from the norm. In summary, the results of this study support the factorial validity, reliability and discriminating validity of the questionnaires.

In all samples, the PRS-AD, TRS-AD and SRS-AD subscales showed satisfactory internal consistencies, regardless of whether only children, adolescents and young adults with anxiety disorders or also participants with other diagnoses were considered. This is in agreement with the results of Doepfner et al. (2008) based on a community sample of children and adolescents and their parents, and with those of Dose et al. (2015) for the parent and child versions in a smaller clinical sample of children and adolescents with anxiety disorders. However, the Specific Phobias subscale rated by children or adolescents themselves demonstrated only low internal consistency both in the sample of children, adolescents and young adults with an anxiety disorder and in the sample including all diagnoses; the Specific Phobias subscale rated by parents showed low internal consistency in the sample of participants with anxiety disorders. This is consistent with previous research on the SRS-AD and the PRS-AD in community and clinical samples (Doepfner et al. 2008; Dose et al. 2015) and with research on the psychometric properties of the SCAS and the SCARED (e.g., Arendt et al. 2014; Bodden et al. 2009; Essau et al. 2002; Ishikawa et al. 2009; Muris et al. 2000; Muris and Steernemann 2001; Nauta et al. 2004; Orgilés et al. 2012; Spence et al. 2003). One reason for this finding may be that the construct of specific phobias as defined by DSM covers a wide range of fears. For clinical practice, we recommend that the individual items of the Specific Phobias subscale should be examined and any striking values used as a starting point for a further exploration.

In the child sample including participants with all diagnoses and in the sample of parents of anxious children, adolescents and young adults, item 25 (“afraid of animals”) demonstrated a low item–subscale correlation. This item might be phrased too vaguely as a child could be afraid of some (or a few) animals, but not of others. Similarly, item 31 (“is afraid of other situations”) showed low item–subscale correlations in both the child and parent samples when considering participants with all diagnoses and in the child sample when only considering anxious participants. Again, a possible explanation might be that this item covers a wide range of different situations. Deletion of this item did not substantially improve the internal consistency in any sample. Item 31 was retained in the scale as it hints on specific phobias that are not directly covered by other items of the scale. If striking values for this item occur, the clinician should explore which “other situations” specifically evoke fears. Use of a rating scale that assesses specific phobias, such as the “Fear Survey Schedule for Children – Revised” (Ollendick 1983; German version: “Phobiefragebogen fuer Kinder”; Doepfner et al. 2006), may be helpful in such cases.

For all samples including participants with anxiety disorders and other diagnoses, a first-order model with four (in the child/adolescent and parent samples) or three (in the teacher sample) correlated factors was superior to a unidimensional solution. In addition, we found evidence that a second-order factor may explain the correlations between the first-order factors; that is, the analyses produced low residual correlations for the second-order model in the child/adolescent and parent samples and substantial factor loadings of the first-order factors on the second-order factor in all samples. Second-order models cannot show a better model fit than first-order models, but aim to provide a more parsimonious account for the correlations between the first-order factors (Brown 2006). Hence, we adopted the second-order model for all three rating scales. For practical use of the scales, the fit of a second-order model hints on the interpretability of subscale scores as well as a total score.

Since the confirmatory factor analysis produced a CFI in the borderline range and an unsatisfactory χ2/df-value in the parent sample, some modifications were made to the second-order model of the PRS-AD. We allowed correlations between the residuals of some items covering sleep-related anxieties and cross-loadings of several items on factors they were contextually related to (see results section). When the same modifications were applied to the second-order models for the SRS-AD and the TRS-AD, model fit also improved substantially.

However, if modifications are made to a model, the factor analysis loses its confirmatory character and becomes exploratory (Brown 2006). Therefore, the newly found modified factor structures have to be validated in other samples to be confirmed. Nevertheless, these results provide support for a factor structure largely corresponding with DSM-diagnoses of anxiety disorders in a clinical sample of children or adolescents, their parents and teachers.

When we only considered children, adolescents and young adults with anxiety disorders for the confirmatory factor analyses, the first-order model and the second-order model provided a satisfactory fit for the PRS-AD and the SRS-AD and the first-order model provided a satisfactory fit for the TRS-AD. Again, we found substantial loadings of the first-order factors on a second-order factor for the parent and the teacher sample and low residual correlations for the second-order model in the parent sample, thus supporting a second-order solution and hinting on the interpretability of both subscales and a total score for the PRS-AD and the TRS-AD. However, the residual correlations for the second-order model in the sample of children, adolescents and young adults themselves did not support the applicability of this model, although the loadings of the first-order factors on the second-order factor were also substantial in this sample. Hence, the results of this study do not clearly favour a first-order or a second-order solution for the SRS-AD.

The total scores of the SRS-AD and PRS-AD discriminated well between children/adolescents with anxiety and non-anxiety disorders. However, for some subscales we did not observe any significant differences between the groups with and without anxiety disorders. A possible reason may be that the anxiety disorders group included a range of different anxiety disorders. For the Generalized Anxiety scale rated by parents, we even found that the non-anxious group showed higher ratings. This may be because symptoms covered by this scale (e.g., nervousness, concentration problems, irritability) may also be present in children and adolescents with other psychiatric disorders. Moreover, for the total score of the TRS-AD rated by teachers, the difference between groups was nonsignificant, i. e., this scale did not discriminate well between anxious and non-anxious children, adolescents and young adults. An explanation may be that teachers often spend limited time with their students and, thus, may not get a comprehensive impression of their anxiety. This conclusion is supported by the finding that children and adolescents and their parents reported more severe symptoms than teachers, and points out the necessity to include different informant perspectives into the diagnostic process.

As hypothesized and consistent with previous research on DSM-based scales for the assessment of anxiety disorders (e.g., Arendt et al. 2014; Brown-Jacobsen et al. 2011; Cosi et al. 2010; Nauta et al. 2004; Weitkamp et al. 2010; Wren et al. 2004), we detected only low-to-moderate agreement between the ratings of different informants. Again, this finding underlines the importance of involving different informants in the diagnostic process (Achenbach et al. 2008). In case of differing ratings by different informants, reasons for these should be explored to decide how to deal with the observed differences (e.g., different information basis for rating internalizing problems, simulation or dissimulation, situation-specific appearance of anxieties; see Doepfner et al. 2008).

We found only low correlations between the subscales and total scales of the different questionnaires and the age of the children, adolescents and young adults, regardless of whether only participants with an anxiety disorder or also participants with other diagnoses were considered. However, significant gender differences emerged on several scales of the PRS-AD and the SRS-AD when regarding the complete sample of participants with anxiety or other disorders, with higher scores detected for females. This is in agreement with the higher prevalence of anxiety disorders in girls found in epidemiological studies (Essau et al. 2000). The finding that this difference did not appear in the teacher ratings might again be due to the fact that teachers may not have sufficient information to rate their students’ anxieties properly.

Limitations

The study has some limitations. First, as the data for the analyses were collected in routine clinical care, interrater reliability of the diagnoses, which were made based on a semi-structured clinical interview, could not be checked. However, the internal consistencies of the scales derived from this checklist are mostly acceptable (Goertz-Dorten and Doepfner 2008). Moreover, for each patient diagnoses were discussed with and confirmed by a supervisor. Second, we were not able to include healthy control children and adolescents in the analyses on the scales’ discriminating validity and the diagnoses in the non-anxious control sample (as well as in the total sample used for all analyses) were very heterogeneous. This might have biased the results. Third, as we used a clinical sample of youths who were referred to our clinic and did not actively recruit youths with specific disorders, our sample included only few participants with a generalized anxiety disorder. This might be problematic as one of the questionnaires’ subscales specifically assesses this disorder. Fourth, we did not collect information about whether the rating scales in the parent sample were completed by mothers, fathers, or other relatives. From our practical experience, we suppose that most of the ratings were done by mothers. Due to this missing information, we could not provide separate results for the ratings of mothers and fathers and examine potential differences between these ratings. Finally, the convergent and divergent validity of the PRS-AD, TRS-AD and SRS-AD remain to be examined.