Introduction

The World Health Organization suggests that screening is justified when a disorder poses a significant public health problem, has a recognizable early symptomatic stage, acceptable approaches to treatment, and suitable screening and diagnostic tests that are cost-effective [1]. These conditions appear to be satisfied in the case of Anxiety Disorders. Anxiety Disorders (AD) are remarkably common in primary-care pediatric settings ranging from 1 to 10 % [2, 3] but are mostly undetected and undertreated [4, 5]. As anxiety symptoms may be mild and transient, may be considered a normal part of adolescent development, is often masked by the co-morbidities with variations in presentation the recognition of clinically significant anxiety disorders is compromised [6, 7]. These AD contributes to poor school performance, somatic manifestations, and functional impairment and therefore needs to be recognized as early as possible [8–10].

One way to improve the case identification in primary-care settings is to use measures validated for the population in the community and culture. To screen for AD in primary-care settings among adolescents many self-rated measures exist. Most widely used measures are Social Anxiety Scale for Adolescents, Social Phobia and Anxiety Inventory for Children, Phobia Questionnaire for Children and Adolescents, Children Anxiety Test, Spence Children’s Anxiety Scale and Revised Child Manifest Anxiety Scale. Although several measures have been developed to evaluate anxiety disorder symptoms, Screen for Child Anxiety Related Emotional Disorders (SCARED) has been found to be one of the best self-report instrument to evaluate anxiety symptoms [6]. Recently, a meta-analysis has documented that the SCARED can be utilized as a screening instrument for DSM-IV-TR anxiety disorder for adolescents from various countries [11]. Of the 35 studies that have documented the various psychometric properties of SCARED across cultures, two studies have been conducted in South Africa and one study in China. SCARED has not been validated for the subcontinent population or other African or Asian cultures. Therefore, the aim of this study was to evaluate the diagnostic accuracy, reliability and validity of SCARED in another non-Western culture before adopting the measures for clinical and research use in India.

Material and Methods

Only the methodological aspects relevant to validation of SCARED are mentioned here. The detailed methodology is described in the accompanying paper on rationale and study design.

Setting and Population

This study was done prospectively in a semi-urban community from April through May 2010. India has the world’s biggest Integrated Child Development Program (ICDS). The smallest functional unit of the ICDS is the anganwadi. The ICDS supervisors and anganwadi workers were sensitized about the project in a two-day workshop and they recruited the adolescents for the study from the Pattanakad ICDS block, of Allapuzha district in Kerala if the adolescent satisfied the selection criteria. The selection criteria were: (1) being in 11 and 19 y adolescent age group; (2) could read Malayalam or English to respond to the self-rated measures; (3) and availability of a primary-care giver staying with the participant to supplement the information required for socio-economic as well as demographic data; (4) voluntary participation after written informed consent from the care-giver and verbal assent from the adolescent. Thus five hundred adolescents from 27 anganwadi areas of Thuravoor panchayath of Pattanakad ICDS block were found suitable for the study. The eligible adolescents and primary care givers were informed about the survey by the respective anganwadi worker 3 to 4 d prior to the data collection. The data were collected with the following measures.

Measures

The Screen for Child Anxiety Related Emotional Disorders (SCARED) [12] is a self-rated questionnaire for measuring anxiety symptoms in children and adolescents. The measure has 41 items under the five subscales of panic/somatic, generalized anxiety, separation anxiety, social phobia, and school phobia. Adolescents are asked to rate the frequency with which they experience each symptom using a, three-point, likert scale (0 = almost never, 1 = sometimes, and 2 = often). Four of these subscales measure anxiety disorder symptoms as conceptualized in the DSM-IV-TR: panic disorder, generalized anxiety disorder, separation anxiety disorder, and social anxiety. The fifth subscale, school anxiety, represents a common anxiety problem in childhood and adolescence. Summing of all the items provides the SCARED total score and if the score is ≥31, the adolescent is considered to have an anxiety disorder in other cultures [12, 13]. SCARED-child version was the measure for studying the validation in this study.

The K-SADS-PL is a semi-structured interview designed to assess 32 psychiatric disorders in children and adolescents based on DSM-IV-TR diagnostic criteria [14]. All adolescents were assessed for present and lifetime episodes of psychopathology using section of the K-SADS-PL (anxiety disorders) to generate DSM-IV-TR diagnosis of Anxiety Disorders. The original version of K-SADS-PL was reported to have good test–retest and inter-rater reliability. This DSM-IV-TR diagnosis was used to measure the criterion validity of SCARED and for evaluating its diagnostic accuracy.

Data Collection

The developmental therapists, special educators, speech therapists, clinical psychologist and pediatrician (N = 8) experienced in assessing children and adolescents, collected the data from the adolescents. They underwent a three-day training workshop in using the instruments prior to data collection and the final inter-rater reliability (with intra class correlation coefficient) for SCARED was 0.86. Following their training, the raters met the adolescents and the primary care givers at the premises of the respective anganwadi, explained about the study and obtained written informed consent from the primary caregiver and verbal assent from the adolescent for participating in the study. Face-to-face interviews were carried out with all the adolescents and their primary care-givers privately and separately. One of the raters administered the self-rated measures to the adolescents individually and the other rater interviewed the primary care-giver independently to obtain supplementary information and for socio-demographic details. Each interview took about 2 h to complete. Participants were told that confidentiality would be broken when someone posed imminent risk of serious harm to themselves or others. Adolescents noted to have anxiety disorder, depressive disorder or suicidal risk, clinically, were referred to the nearest medical college hospital for treatment. The local Institutional Review Board of the Child Development Centre had reviewed and provided approval for the study.

Data Analysis

Sensitivity, specificity, likelihood ratios and predictive values as the parameters for diagnostic accuracy for various total SCARED scores was calculated against the gold standard diagnosis of DSM-IV-TR diagnosis for Anxiety Disorder using the Receiver Operating Characteristic (ROC) curve. Similarly, the various diagnostic accuracy parameters for various SCARED subscale cut-off scores were calculated against the respective DSM-IV-TR. In order to determine the optimal screening threshold ROC curve and contingency tables were used. The nominal Area Under the Curve (AUC) was computed to evaluate the overall diagnostic efficiency of the tests. To assess the reproducibility, the test-retest reliability and the inter-rater reliability of SCARED were examined with the intra class correlation. For internal consistency, Cronbach’s α coefficient was calculated for the total scale and subscales. To identify the items that contribute to and discriminate between children who score high and low on the total set of items, the authors performed an item-total correlation. To determine the criterion validity of SCARED as a measure of anxiety disorders, the dichotomized SCARED score (score of 21 deciding the ‘caseness’) was compared with the DSM-IV-TR clinical diagnosis of Anxiety Disorders. The concordance (overlapping cases) of the DSM-IV-TR diagnosis of anxiety disorders and SCARED diagnosis of anxiety disorder was computed as the quotient of the cases classified as anxiety disorder by both the measures and the number of cases classified as anxiety disorder by neither of the measures. Cohen’s kappa (unweighted) was also calculated to assess the agreement between the DSM-IV-TR diagnosis and dichotomized total SCARED score. Data was analyzed using SPSS (version 19).

Results

Diagnostic Accuracy

The cut-off SCARED score of ≥21 to identify any anxiety disorder had a sensitivity of 84.62 (95 % CI of 65.1 to 95.5), specificity of 87.36 (95 % CI of 81.5 to 91.9), positive predictive value of 50(95 % CI of 34.4 to 65.6), negative predictive value of 97.4 (95 % CI of 93.6 to 99.3), positive likelihood ratio of 6.69 (95 % CI of 5.6 to 8) and negative likelihood ratio of 0.81 (95 % CI of 0.07 to 0.5). The area under curve (AUC) in the ROC for the SCARED was 0.90 (95 % CI = 0.86 to 0.94; z = 10.51; P = 0.0001).

The cut-off for the Panic Disorder subscale score of ≥5 to identify Panic Disorder had a sensitivity of 63.64 (95 % CI of 30.8 to 89.1), specificity of 86.24 (95 % CI of 80.5 to 90.8), positive predictive value of 21.2 (95 % CI of 8.8 to 39.2), negative predictive value of 97.6 (95 % CI of 94 to 99.3), positive likelihood ratio of 4.63 (95 % CI of 2.9 to 7.3) and negative likelihood ratio of 0.42 (95 % CI of 0.2 to 1). The AUC in the ROC for this subscale was 0.86 (95 % CI = 0.81 to 0.91; z = 6.96; P = 0.0001).

The cut-off for the Generalized Anxiety Disorder subscale of ≥7 to identify Generalized Anxiety Disorder had a sensitivity of 85.71 (95 % CI of 42.1 to 99.6), specificity of 93.26 (95 % CI of 88.8 to 96.4), positive predictive value of 31.6 (95 % CI of 12.6 to 56.6), negative predictive value of 99.4 (95 % CI of 97 to 100), positive likelihood ratio of 12.73 (95 % CI of 9.4 to 17.3) and negative likelihood ratio of 0.15 (95 % CI of 0.02 to 1). The AUC in the ROC for this subscale was 0.97 (95 % CI = 0.94 to 0.99; z = 34.05; P = 0.0001).

The cut-off for the Separation Anxiety Disorder subscale of ≥8 to identify Separation Anxiety Disorder had a sensitivity of 66.67 (95 % CI of 22.3 to 95.7), specificity of 95.36 (95 % CI of 91.4 to 97.9), positive predictive value of 30.8 (95 % CI of 8.5 to 62.8), negative predictive value of 98.9 (95 % CI of 96.2 to 99.9), positive likelihood ratio of 14.37 (95 % CI of 8.2 to 25.3) and negative likelihood ratio of 0.35 (95 % CI of 0.10 to 1.3). The AUC in the ROC for this subscale was 0.77 (95 % CI = 0.71 to 0.83; z = 1.81; P = 0.06).

The cut-off for the Social Anxiety Disorder subscale score of ≥8 to identify Social Anxiety Disorder had a sensitivity of 80 (95 % CI of 44.4 to 97.5), specificity of 91.05 (95 % CI of 86.1 to 94.7), positive predictive value of 32 (95 % CI of 14.6 to 54), negative predictive value of 98.9 (95 % CI of 95.9 to 99.9), positive likelihood ratio of 8.94 (95 % CI of 6.5 to 12.2) and negative likelihood ratio of 0.22 (95 % CI of 0.06 to 0.8). The AUC in the ROC for this subscale was 0.92 (95 % CI = 0.87 to 0.95; z = 9.60; P = 0.0001). The ROC curves for the total SCARED scores and subscales against any DSM-IV-TR Anxiety Disorder diagnosis and respective specific Anxiety Disorder are depicted in Fig. 1.

Fig. 1
figure 1

The Receiver Operating Curve characteristics of SCARED and its subscales. a is the ROC curve for the SCARED total score against any DSM-IV-TR Anxiety Disorder diagnosis; b is the ROC curve for the SCARED Panic Disorder subscale score against any DSM-IV-TR Panic Disorder diagnosis; c is the ROC curve for the SCARED Generalized Anxiety Disorder subscale score against any DSM-IV-TR Generalized Anxiety Disorder diagnosis; d is the ROC curve for the Separation Anxiety Disorder subscale score against any DSM-IV-TR Separation Anxiety Disorder diagnosis; e is the ROC curve for the SCARED Social Phobia subscale score against any DSM-IV-TR Social Phobia

Full size image

Reproducibility

The test-retest reliability at one year was studied to assess the reproducibility of SCARED and the intra class correlation coefficient was found to be ICC = 0.90. The inter-rater reliability of SCARED as measured with intra class correlation coefficient was also found to be ICC = 0.87. The details of the reproducibility are mentioned in Table 1.

Table 1 Inter-rater reliability and test-retest reliability at the SCARED scale and subscale level
Full size table

Validity

The Cronbach’s alpha coefficient for the entire scale was 0.89 and the internal consistency of the various items, ranged from 0.05 to 0.95, and the internal consistency of the subscales are mentioned in Table 2. The item -total correlation details too are presented in Table 2.

Table 2 Internal consistency and total-subscale correlation of SCARED and subscales
Full size table

The assessment of the criterion validity revealed that there was a high concordance rate of 81 % (DSM-IV-TR any Anxiety Disorder vs. SCARED = 72/500 vs. 129/500) [Cohen’s κ = 0.42 (95 % CI = 0.31 to 0.52); P = 0.001] between the SCARED and reference standard of DSM-IV-TR diagnosis in identifying any Anxiety Disorder among the children.

Discussion

This study assessed the diagnostic accuracy, reproducibility and validity of a Malayalam translation of the Screen for Child Anxiety Related Emotional Disorders (SCARED) in a representative sample of adolescents in a community. While the SCARED uses the well-tested Anxiety Disorders symptom list, it still had to be evaluated empirically before field use in another culture with different psychosocial constructs and therefore was done in the present study. Thus the present study is the first to document the psychometric properties of diagnostic accuracy, reliability and validity of SCARED in the Indian population.

The threshold SCARED total score of 21 for screening of Anxiety Disorders, using ROC cut-off, was different from the originally suggested cut-off score of 31 [12]. A threshold score of ≥21 in SCARED was considered ideal as a screening cut-off score to identify possible cases of Anxiety Disorders. This lower cut-off score has better diagnostic accuracy properties compared to the original validation study values and in other studies [15, 16]. This difference in scores could be because of the adolescent population in the present study versus children with Anxiety Disorders in the previous studies or because the previous studies used clinical samples [15, 16]. The AUC associated ROC curve for a screening is considered a measure of the overall diagnostic efficacy of the test. In the index study the AUC 0.90 (P = 0.0001) suggests that the overall diagnostic accuracy of SCARED is excellent for a SCARED score of ≥21. The AUC for all the subscales were also excellent except for the Separation Anxiety Disorder high, which was 0.77 (95 % CI = 0.71 to 0.83; z = 1.81; P = 0.06). SCARED (score of ≥21) was also found to be as accurate as DSM-IV-TR in the detection of Anxiety Disorders thus demonstrating a high concordance rate.

The authors have demonstrated that the Malayalam version of SCARED possesses satisfactory internal consistency. The α coefficient for the whole scale was 0.89 and is similar to the 0.90 that was reported with the original, 41-item, English version [12] and 0.91 reported with the German version [17] but lower than the 0.93 reported in a North American rural community sample recently [18]. The internal consistency for the subscales in the German translation ranged from 0.66 to 0.81 and is comparable with the present values of 0.68 to 0.80. A recent meta-analysis has reported an overall alpha coefficient of 0.54 to 0.86 for the various versions of SCARED [11].

The test-retest reliability for the whole SCARED scale reported in the present study was 0.90, much higher than the 0.70 documented elsewhere [18]. The test-retest reliability of the subscales also was high (ranging from 0.76 to 0.85) suggesting that SCARED and its subscales possess adequate temporal stability when used in this rural adolescent population. The inter-rater reliability of SCARED was 0.87, and thus was high, suggesting that SCARED measures the same construct when used by different raters. Comparison with other studies is not done as such data are not readily available in the literature.

In conclusion, the present study demonstrates that the SCARED is a valid instrument to assess Anxiety Disorders in Indian adolescents from the general community. Future validation of SCARED in the Indian context should study the factor structure of anxiety and SCARED, as well as the convergent and divergent validity of the measure.