Introduction

Separation anxiety disorder (SAD) is characterized by excessive and inappropriate anxiety experienced by a child when separation from the home or attachment figures (generally the parents) occurs or is anticipated [1]. It is one of the most prevalent anxiety disorders in childhood: 3.9 % in children aged 6–12 years and 2.6 % in adolescents aged 13–18. The mean age at onset of childhood anxiety disorders is 8 years [2]. The avoidance of situations involving separation from attachment figures or the home seriously restricts the child’s social relations, has negative repercussions for family functioning and causes problems of school attendance. Among cases of school refusal, a significant proportion have a diagnosis of SAD: 22.2 % in the clinical population [3] and 10.8 % in the community population [4]. Comorbidity with other disorders is very high (86 %) [5], especially with generalized anxiety disorder (74 %) and with specific phobia (58 %) [6]. SAD is a risk factor for numerous disorders, and particularly for panic disorder and depression [7–11]. With regards to adults with SAD, 36.1 % presented it in childhood, more commonly women [12].

The most widely used interview for diagnosing separation anxiety disorder is the Anxiety Disorders Interview Schedule for DSM-IV Child/Parent version (ADIS-IV-C/P; [13]), which also includes a section on school refusal. Other interviews include the Schedule for Affective Disorders and Schizophrenia for School Aged Children (K-SADS; [14]) and the Diagnostic Interview for Children and Adolescents (DICA; [15]). Pictorial interviews, such as the Dominic Interactive Assessment (DIA; [16]) and the Separation Anxiety Test (SAT; [17, 18]), use photographs and cartoon figures.

Self-reports are useful instruments in epidemiological and clinical studies given their ease of administration and low cost. For internalizing disorders, children are better informants of subjective distress, as parents tend to underestimate it. Longitudinal studies reveal that lifetime diagnosis is more likely to concur when the informant is the child [19]. The main global self-rating scales are the Multidimensional Anxiety Scale for Children (MASC; [20]), the Screen for Child Anxiety Related Emotional Disorders—Revised (SCARED-R; [21]) and the Spence Children’s Anxiety Scale (SCAS; [22]). All of these three include a brief subscale on SAD that is useful as a screening instrument.

The self-rating scales designed specifically for assessing childhood separation anxiety [23–27] provide more detailed information on the disorder. The pioneering instrument among these is the Separation Anxiety Assessment Scale (SAAS), based on the clinical child literature, DSM-IV criteria, structured interviews, child daily diaries of separation anxiety events and the authors’ clinical work experience. The SAAS is made up of four key symptom dimensions (plus two complementary subscales) that discriminate between anxiety disorders. However, the authors’ theoretical proposal was not validated. In the study that used the original American version [28], the participants were 111 children and adolescents aged 6–17 diagnosed with at least one anxiety disorder; the principal components factor analysis found two factors, explaining 44.39 % of the variance. The study that used the Italian adaptation [29] of the test was carried out with 358 children aged 6–10; the principal components factor analysis with varimax rotation identified three factors that explained 42.78 % of the variance. The aim of the present study was to analyze the psychometric properties of the Spanish adaptation of the SAAS, using a large community sample, and with a particular focus on examining the factorial structure of the four key symptom dimensions.

Study 1: Exploratory Factor Analysis

Method

Participants

We carried out a procedure of random sampling by clusters, the primary units being the districts of two provinces in the south of Spain, the secondary units being the schools and the tertiary units the classrooms. Recruitment took place in 14 schools, providing a total of 1343 children from primary school grades 3–6. Of these, 33 (2.46 %) were excluded due to errors or omissions in their responses, 17 (1.27 %) because they were foreigners with insufficient command of the Spanish language, and 12 (.89 %) due to failure to obtain the parents’ informed consent. The final sample was made up of 1281 schoolchildren, with an age range of 8–11 years (M = 9.57; SD = 1.14) and 51.05 % girls. The Chi-square test of homogeneity of the distribution of frequencies revealed that there were no statistically significant differences among the eight groups of gender by age (χ2 = 4.25, p = .24). The participants were from a broad socio-economic range, determined according to the type of school (public, grant-assisted or private) and its location (city or town, village, rural).

Procedure

The school management teams were contacted and the aims of the research explained. Once the school had agreed to participate in the study, all parents were sent an informative letter and a request for written informed consent. The self-report was administered collectively in the classroom during normal school hours. The researchers read the instructions aloud, dealt with any questions or doubts, and supervised the filling out of the instrument.

Measure

With the authors’ permission, the SAAS was translated into Spanish using the back-translation method [30]. It comprises four key symptom dimensions: fear of being alone (FBA), including fear of being in a room alone even if one or both parents are at home, fear of abandonment (FAb), that is, fear of being abandoned by one’s parents in different situations, fear of physical illness (FPI), meaning fear of somatic symptoms, and worry about calamitous events (WCE)—worrying about something bad happening.

Additionally, the instrument includes the complementary subscales Safety Signals Index (SSI), referring to persons, places, objects and actions that provide security, an example item being item 10: “How often do you need your mom or dad to promise to stay at home so that you can go to a play date, birthday party, or after-school activity?”, and Frequency of Calamitous Events (FCE), referring to stressful life events that can trigger or exacerbate separation anxiety, such as in item 5 “How often has a parent, family member, friend, or relative been in a serious accident?”

The SAAS is made up of 34 items, five in each subscale except the SSI, which has nine. Items are scored using the following four-point Likert-type scale (specifying the number of times in the case of the FCE subscale): never = 1, sometimes (once) = 2, most of the time (twice) = 3, all the time (three or more times) = 4.

Data Analysis

In order to explore the underlying structure of the four key symptom dimensions, an iterative principal axis factor analysis with oblimin rotation was carried out with the corresponding 20 items, using the statistics program SPSS 20.0.

Results

The criteria for obtaining the factor solution were retaining the factors with an eigenvalue greater than 1 (Kaiser criterion) and assigning to each factor the items that loaded over .30. The Kaiser–Meyer–Olkin measure of sampling adequacy (KMO = .89) and the Bartlett sphericity test [31, 32] [χ2(N = 190) = 6053.01, p = .00] yielded adequate values.

The factor structure obtained was the same as that expected (Table 1), explaining 47.77 % of the variance: Factor 1, worry about calamitous events (13.41 %); Factor 2, fear of abandonment (12.90 %); Factor 3, fear of being alone (11.48 %); and Factor 4, fear of physical illness (9.98 %). Items 2, 3 and 19 presented loadings of between .30 and .40 in two factors. It was decided to assign them to the factor in which the loading was greatest, a strategy that also coincided with the theoretical approach of the instrument’s authors. The wording of items 2 and 19 is different from that of the other items of their respective factors: “How often are you afraid to…?”, so these items should perhaps be rewritten. Item 3, “How often do you worry about getting picked up late from school, a party, or another activity?”, can equally well be included in WCE, as the authors propose, or in FAb, since it is not clear whether the most relevant content is the worry or the situation of being away from home without one’s parents. This item could be substituted with a worry that is more characteristic of separation anxiety, such as worrying about one’s parents having an accident.

Table 1 Exploratory factor analysis
Full size table

Study 2: Confirmatory Factor Analysis, Reliability and Validity

Method

Participants

We carried out a procedure of random sampling by clusters similar to the one in Study 1. In 51 schools, we recruited 4858 Primary School pupils from grades 3–6, of whom 134 (2.76 %) were excluded due to errors or omissions in their responses, 59 (1.21 %) because they were foreign students with serious deficiencies in their command of Spanish, and 37 (.76 %) due to a lack of informed consent to participate from their parents. The final sample was made up of 4628 children aged 8–11 (M = 9.60; SD = 1.13), 48.92 % of whom were girls. The Chi square test of homogeneity of the distribution of frequencies showed that there were no statistically significant differences among the eight groups of gender by age (χ2 = 7.14, p = .07). The socio-economic status of the participants was similar to that of Study 1. The test–retest reliability and diagnostic validity were calculated with 1726 and 392 children, respectively, chosen at random from the sample.

Procedure

The process of providing information, asking for permission, promoting participation and applying the self-reports was similar to that of Study 1. The children filled out, collectively and in the classroom, a battery of four self-reports, in random order for each class group (20–25 students). The retest students filled out the SAAS again 3 weeks after the first application and, in order to calculate the diagnostic validity, the participants assessed were interviewed individually through a semi-structured interview based on the DSM-IV criteria.

Measures

  1. 1.

    The SAAS [23]. See Study 1.

  2. 2.

    The Children’s Separation Anxiety Scale (CSAS; [27]). This instrument assesses the frequency of separation anxiety symptoms in children aged 8–11. It comprises 20 items scored on a five-point scale, from never or almost never (1) to always or almost always (5). It is made up of four factors that explain 46.91 % of the variance: Factor 1, Worry about separation; Factor 2, Discomfort from separation; Factor 3, Opposition to separation; Factor 4, Calm at separation. The internal consistency (Cronbach’s alpha) is .82 and the test–retest reliability is .83.

  3. 3.

    The Childhood Anxiety Sensitivity Index for Children (CASI; [33]). We used the Spanish adaptation for children aged 9–11 [34]. It assesses fear of sensations related to anxiety, and comprises 18 items scored on a three-point scale: 1 = none, 2 = some, 3 = a lot. It has a multidimensional structure (somatic, mental, and control/social). Its internal consistency is high (α = .89).

  4. 4.

    The School Fears Survey Scale-Form II (SFSS-II; [35]). This tool assesses fear and anxiety in school situations among children aged 8–11, and comprises 25 items scored on a three-point scale: 0 (not at all), 1 (a little), 2 (a lot). It has four factors that explain 55.80 % of the variance: Factor 1, Fear of academic failure and punishment; Factor 2, Fear of physical discomfort; Factor 3, Fear of social and school assessment; Factor 4, Anticipatory and separation anxiety. The internal consistency (Cronbach’s alpha) is .89.

  5. 5.

    The Anxiety Disorders Interview Schedule for Children for DSM-IV (ADIS-IV-C; [36]). We used the Spanish adaptation [37] for children and adolescents aged 7–17. The interviewers were psychologists specifically trained by means of a manual [38]. The ADIS-IV is a highly appropriate instrument for the assessment and diagnosis of SAD [39]. In the present study the kappa coefficient for SAD was 1 (perfect agreement).

Data Analysis

The analyses were carried out with the 20 items of the four key symptom dimensions. The structure obtained in Study 1 was examined by means of confirmatory factor analysis. The internal consistency was calculated with Cronbach’s alpha coefficient. We carried out a classical analysis of items through the correlations of the items with the corresponding factor and with the total score. The test–retest reliability and concurrent validity were calculated with the Pearson product-moment correlation coefficient. A receiver operating characteristic (ROC) curve was created and the area under the curve (AUC) was calculated to study the sensitivity and specificity. In addition, we examined the validity and reliability of the two complementary subscales. The analyses were carried out using the statistics packages SPSS version 20.0, AMOS version 20.0 and MedCalc version 12.5.

Results

Confirmatory Factor Analysis

Four alternative models were evaluated (see Table 2): (1) the null or independent model, which presupposes maximum independence between the items or an absence of factor structure (M0); (2) the one-dimensional model, in which the 20 items were forced to load in a general factor of separation anxiety (M1); (3) the four uncorrelated factors model (M4); and (4) the four correlated factors model (M4*), which was expected to show the best goodness-of-fit indices. The models were examined by means of the following goodness-of-fit measures: the Root Mean Square Error of Approximation (RMSEA), the Goodness of Fit Index (GFI), the Adjusted Goodness of Fit Index (AGFI), the Normed Fit Index (NFI), the Comparative Fit Index (CFI), and the Tucker-Lewis Index (TLI), as well as the Chi-square statistic (χ2). For the RMSEA a value of less than .05 [40] is recommended, and for the NFI, CFI and TLI the value suggested is .95; it is also recommended that a combination of fit indices be used in order to reduce both type I and II errors [41]. For the GFI and the AGFI, values greater than .90 are considered acceptable.

Table 2 Fit indices for confirmatory factor models
Full size table

The Chi-square statistic was significant, indicating a poor fit for all the models. However, it should be borne in mind that this statistic is influenced by sample size, so the differences may be significant with large samples even though the models fit the data well. The only model with adequate goodness-of-fit index values was that of the four correlated factors.

Table 3 shows the correlation coefficients among the factors and with the total score in the four factors.

Table 3 Correlation coefficients among factors and with the total score in the four factors
Full size table

Internal Consistency and Item Analysis

The internal consistency coefficient (Cronbach’s alpha) was good for the total score of the four factors of the SAAS (.84), adequate for WCE (.75), FAb (.73) and FBA (.71), and low for FPI (.60). The range for the item-factor correlations was .48 to .78, and that of the item-test correlations was .37 to .58, indicating adequate behaviour of all the items.

Test–Retest Reliability

The test–retest reliability coefficient was good for the total score of the four factors of the SAAS (r = .77) and adequate for the factors: WCE (r = .69), FAb (r = .65), FBA (r = .65), and FPI (r = .68).

Convergent Validity

We calculated the correlation coefficients of the factors and of the total score with another separation anxiety self-report and with two self-reports that assess related constructs (Table 4). The correlation between the SAAS and the CSAS was high (r = .71). The correlation between the factors generally followed the expected pattern: WCE was especially related to Worry about separation, FAb and FBA to Discomfort from separation and Opposition to separation, and FPI to Discomfort from separation. The correlation coefficients of the SAAS factors with the Calm at separation factor of the CSAS were low and negative.

Table 4 Correlation coefficients of the SAAS with other self-reports
Full size table

The SAAS correlated more strongly with the CASI (r = .60) than with the SFSS-II (r = .48), particularly in relation to the respective somatic (r = .56) and fear of physical discomfort (r = .42) factors of these instruments.

Discriminant Validity

Table 5 presents the score for each of the four factors and for the total of these four factors of the children diagnosed with SAD (N = 32), excluding the cases of comorbidity. The score of the children with SAD was higher on FAb and FBA, fears that are specific to this anxiety disorder.

Table 5 The scores of the scale with childhood anxiety disorders
Full size table

Sensitivity and Specificity

We operationalized sensitivity as the proportion of children with SAD diagnosed through the ADIS-IV-C that were correctly classified by means of the total score in the four factors of the SAAS. Likewise, the specificity was indicated by the percentage of children with SAD according to the interview that were correctly identified with the total score in the four factors of the SAAS. A receiver operating characteristic curve (ROC) and the area under the curve (AUC) were examined to establish the optimal cut-off score, and it was found that the AUC for ROC for the cut-off of 41 was .92 (95 % CI .89–.95).

The Youden Index reflects the difference between the rate of true positives and false positives, such that the closer it is to 1, the better the diagnostic capacity. The results showed that a score of 41 is the optimal cut-off point with good sensitivity (83 %; 95 % CI .89–.95) and specificity (93 %; 95 % CI .89–.95). For the cut-off point of 41, the Youden Index is .78, indicating good diagnostic capacity of the total score in the four factors of the SAAS (Fig. 1).

Fig. 1
figure 1

ROC curves of the SAAS

Full size image

Complementary Subscales

The coefficients of internal consistency and of test–retest correlation were adequate for the SSI (α = .74, r = .71) and low for the FCE (α = .60, r = .57). The correlation of the items with the total score of the corresponding complementary subscale ranged from .44 to .64 for the SSI, and from .44 to .66 for the FCE, indicating the good behaviour of all the items. The correlation with the total score in the four SAAS factors was high for the SSI (r = .72) and low for the FCE (r = .27).

Binary logistic regressions were performed with the complementary subscales as predictors of the SAD diagnosis. The SSI explained 45 % of the variance and correctly classified 83.2 % of the cases; for each unit of increase on the SSI the probability of SAD increased by 43 %. The FCE explained 6 % of the variance and correctly classified 78.9 % of the cases analyzed; for each unit of increase on the FCE, the probability of SAD increased by 21 % (Table 6).

Table 6 Binary logistic regression for SAD diagnosis
Full size table

Discussion

Exploratory factor analysis with the 20 items of the key symptom dimensions of the SAAS identified four related factors, corroborated through the confirmatory factor analysis carried out with a large independent sample: Fear of Being Along (FBA), fear of abandonment (FAb), fear of physical illness (FPI), and worry about calamitous events (WCE). However, the study that used the original American version found a specific factor of separation anxiety, made up of 17 items, and one factor, comprising three items, of somatic complaints/fear of physical illness, common to other anxiety disorders—for example, item 31, “How often are you afraid to eat breakfast at home because you may throw up or choke?” is also characteristic of specific phobia of other types, generalized anxiety or panic. This discrepancy, two or four factors, could be explained by the marked differences between the samples (nature, age, size). Future studies should analyze the factor structure of the SAAS in the American school population and the Spanish school population.

The characteristics of the sample in the Italian adaptation were more similar to those of the present study’s sample, such that the factor structure was similar, except for the fact that FAb and FPI were regrouped into one factor. It would be interesting to ascertain whether the four-dimensional theoretical model, confirmed by the Spanish validation, or the three-dimensional model from the Italian adaptation fits the data better.

The factors FBA and FAb refer to situations that the child tends to avoid because they cause him/her separation anxiety (stimulus variables). The former refers to when the child is alone at home (living room, bathroom, bedroom), especially at night (sleeping alone), which leads him/her to stick closely to his/her parents; the latter refers to when the child is away from home (at a friend or schoolmate’s house, on the school bus) or at home but without the parents (with a babysitter). The avoidance component is common in separation anxiety self-reports. The two factors of the SAAI contain items similar to those of FBA and FAb: Going to school or going to bed alone, for example: “Because I am anxious, I avoid going to sleep alone”, and Being or going home alone when no-one is there, for example: “Because I am anxious, I avoid being alone at home”. Likewise, the factor Opposition to separation of the CSAS includes behaviours for avoiding separation, such as: “Do you protest when your mum or your dad tell you they are going out?” In this regard it would be clearer to change the wording at the beginning of the items of the FBA and FAb factors, from “How often are you afraid to…?” to “How often do you avoid…?”

The factors FPI and WCE constitute the actual experience of separation anxiety (response variables). The former refers to the distress produced by the symptoms characteristic of anxiety (vomiting, choking) and the latter concerns the worry that something bad will happen to one’s parents or oneself. The CSAS has two equivalent factors: Discomfort from separation, for example: “Are you afraid to stay for lunch at school in case you start vomiting or choking?”, and Worry about separation, for example: “Are you worried that something bad might happen to you?” Thus, whilst the SAAI is confined to the behavioural component, i.e., avoiding being alone at home or at school, the SAAS and CSAS also evaluate the psychophysiological and cognitive components of separation anxiety.

The reliability of the total score in the four SAAS factors was good (α > .80, r > .75), and that of the factors adequate (α > .70, r 3 ≥ .65), except in the case of the internal consistency of FPI, which was low (α = .60). These values should be interpreted taking into account the small number of items of the factors.

The SAAS showed a strong relationship with another measure of separation anxiety. Although to a lesser extent, the relationship was good with sensitivity to anxiety, and adequate with school fears, supporting the convergent validity of the instrument. As for the value of the ROC curves, it revealed the diagnostic capacity of the SAAS for differentiating between children with and without SAD.

With regard to the complementary subscales, the SSI emerged as more valid and reliable than FCE. Since safety signals constitute part of the problem, for example, “How often do you need to call your mom or dad so that you can stay home with a babysitter?” (item 18), they predicted the SAD diagnosis better than the potential trigger events, e.g., “How often have there been burglaries in your neighborhood?” (item 16).

According to Lang’s theory [42] separation anxiety is composed of three dimensions: (a) cognitive, or worrying that bad things will happen to the child and/or the parents, (b) psychophysiological, or physical discomfort (feeling sick, choking, etc.), and (c) behavioral, or avoidance of being abandoned and/or being without the parents. The factorial structure of the SAAS confirms this approach, such that WCE corresponds to the cognitive dimension, FPI to the psychophysiological dimension, and FAb and BFA to the behavioral dimension. In this sense, it might be better to replace the sentence “How often are you afraid…?” with “How often do you avoid…?”

As part of the multimethod assessment, the interview, which is necessary for diagnosis [43], is complemented with other assessment tools that help to plan the treatment. According to the review of 50 years of research on childhood anxiety, the most applied treatments include exposure (88 %), cognitive therapy (62 %) and relaxation (54 %) [44]. The SAAS allows you to select the most appropriate therapeutic procedures in each case (Fig. 2).

Fig. 2
figure 2

Prescriptive treatment planning

Full size image

The Spanish validation of the SAAS was carried out using the child him/herself as the sole source of information. It would be useful to validate the version for parents, since SAD affects not only the child but also the attachment figures.

In summary, this study provides data in support of the reliability and validity of the SAAS, a viable tool for screening in epidemiological studies and a useful one for diagnosis, individualized case formulation, and prescriptive treatment planning.

Summary

This study examines the factorial structure and the psychometric properties of the SAAS in two large samples of Spanish schoolchildren. The SAAS is the pioneering self-rating scale designed specifically for assessing childhood separation anxiety. This study adds evidence to the international empirical support for the reliability and validity of the measure. The SAAS can guide the selection of the therapeutic procedures to be applied, for example exposure plus cognitive restructuring if the SAD profile is worry, or exposure plus relaxation training if the profile is physical distress.