Introduction

Impaired social ability is associated with child psychiatric disorders for several reasons. Qualitative abnormalities in reciprocal social interactions are defining characteristics of autistic spectrum disorders (ASDs) [2, 17]. In addition, a broad range of child psychiatric disorders can potentially cause, and be caused by, poor social skills. Thus behavioural and emotional disorders may lead to rejection or neglect by peers [12], reducing opportunities for the social learning of social skills. Conversely, peer problems can increase psychiatric vulnerability to adverse life events or chronic adversities [11].

Even brief assessments of children’s mental health and wellbeing often include items on friendship and popularity [e.g. 1, 10], whereas it is less common for social aptitudes to be assessed directly. This may reflect the implicit assumption that success and failure in peer relationships are themselves adequate indices of good and poor social skills. However, it is worth distinguishing between social inclusion (friendship, popularity, victimization) and social skill because they are conceptually distinct and may be dissociated in practice. Thus, some children with good social abilities have poor peer relationships, for example as the result of racial prejudice. Conversely, some children with poor social abilities have good peer relationships, for instance if they attend schools that work hard to foster the integration of children with disabilities. Ideally, a brief assessment measure of social aptitude would detect deficits in social ability at a young enough age to provide remedial help before these deficits lead to peer problems. Particularly low scores on a measure of social aptitude might also provide useful pointers to unrecognized ASDs.

The Social Aptitude Scale (SAS) is a ten item scale that forms part of the Development and Well Being Assessment [9]. The items were chosen to tap skills in social understanding and behaviour that vary substantially in the general population and that are usually markedly under-developed in individuals with ASDs. The items focus on complex interactive skills rather than on relatively easily coached skills (such as remembering to say “please” and “thank you” or introducing oneself by name). This paper is a preliminary investigation of the psychometric properties of the SAS, based on a national community-drawn sample of children aged 5–16 years.

Method

Sample

The current study was carried out as part of a larger survey of a nationally representative sample of British 5–16 year olds, which is fully described elsewhere [12]. In Great Britain “child benefit” is a universal state benefit payable for each child in the family, and it has an extremely high uptake. For this reason, the British Child and Adolescent Mental Health Survey 2004 used the child benefit register to develop a sampling frame of postal sectors from England, Wales and Scotland. After excluding families with no recorded postcode or subject to current revision of their record, the sampling frame was estimated to represent 90% of all British children [14]. This sampling frame was used to select a stratified random sample of 12,294 families, of whom 9% opted out prior to their details being passed to the investigators, 5% could not be traced and 1% were ineligible [12]. Thus, 10,496 families were approached, of whom, 7,977 responded—representing 65% of those originally selected and 76% of those who were approached. All parents and 11–16 year olds were invited to take part in face-to-face interviews carried out in the family home. When the family consented, a teacher nominated by the family was also mailed a questionnaire; teacher data were available for 74% of the sample.

Measures

The ten item Social Aptitudes Scale (SAS) was completed by parents about their children. The scale includes items such as “Able to compromise and be flexible”, “Easy to chat with, even if it isn’t on a topic that specially interests him/her” and “By reading between the lines of what people say, s/he can work out what they are really thinking and feeling” (see Table 1). For each item, parents rated their child as “a lot worse than average”, “a bit worse than average”, “about average”, “a bit better than average” or “a lot better than average”—the reference group being other children of the same age. The full measure in English and other languages can be viewed and downloaded from http://www.dawba.com/SAS. All ten items were completed on 7,504 individuals (94% of the sample); only these individuals were included in the principal component analysis. Eight or more items were completed on 7,768 individuals (97% of sample); total scores for the remaining analyses were computed for all these individuals (prorating the score when only 8 or 9 items had been completed).

Table 1 Factor loadings and item-score correlations for the 10 SAS items
Full size table

Parents, teachers and 11–16 year olds also completed the Strengths and Difficulties Questionnaire (SDQ) [10]. The SDQ consists of 25 items relating to five symptom scales covering the following areas: emotions; behaviour; attention and activity level; peer relationships; and prosocial behaviour. The first four scales are added together to generate a total difficulties score. The SDQ’s impact supplement asks respondents whether they consider the child to have a significant mental health problem, and if so, enquires further about resultant distress and social impairment—permitting the calculation of an impact score.

The Development and Well-Being Assessment (DAWBA) was used to assess psychiatric disorder [9]. This structured interview was administered by lay interviewers to the parents of 5–16 year olds, and also to the 11–16 year olds themselves, while the interviewers also recorded detailed verbatim descriptions of any problem areas. An abbreviated version was mailed to the teacher. A small team of experienced clinicians used the information provided by all the informants, combining information as they would in the clinic, to make diagnoses according to ICD–10 and DSM-IV criteria [2, 17]. The prevalence of the main diagnostic groupings was as follows: behavioural disorders 5.8%, emotional disorders 3.7%, hyperkinesis 1.5%, ASD 0.9% [12].

Previously published data on children from this sample who had been assigned ASD diagnoses [12] support the validity of these diagnoses. The prevalence of 0.9% is plausible: though higher than the commonly quoted range of 0.3–0.6% [3, 8, 15], it is lower than the prevalence of 1.2% reported by another recent British survey with wide coverage and careful assessment [4]. The mean level of parent-reported impact on the SDQ was higher for children with ASDs than for those with hyperactivity, behavioural or emotional disorders—demonstrating that the high prevalence was not the result of diluting true ASDs with large numbers of children with clinically insignificant symptoms. In addition, there was evidence of divergent validity, with ASDs having very different associations from other disorders, e.g. substantially higher rates of epilepsy and learning disability, but no evidence for the socioeconomic disadvantage associated with other disorders (as judged by parental education, housing tenure and area of residence).

The ASD section of the DAWBA is governed by a “skip rule” so that although all parents are asked some initial questions, only a minority of parents is asked the more detailed questions. The full set of questions was only administered to 744 parents (9.3%) who met one or more of three criteria: the presence of potentially autistic features in the first 3 years that did not resolve completely; an SDQ profile with high peer problems and low pro-social behaviours; or a SAS score of 12 or less. Had the “skip rule” been modified to exclude the SAS criterion, only one of the 67 children in the sample who was diagnosed with an ASD would have been missed. To avoid circularity, this one child was omitted from the analyses when assessing how well the SAS screened for ASDs.

There were no psychometric tests of children’s cognitive abilities or academic attainments. Parents and teachers were asked to estimate each child’s mental age, and teachers reported whether a child had a written statement of special educational needs related to cognitive and learning needs (including specific, moderate, severe and profound learning difficulties, but not distinguishing between them). For the purpose of these analyses, a child was considered to have a generalized learning disability when one or both informants estimated that mental age was 60% or less of the chronological age (e.g. a mental age of 6 or less at a chronological age of 10). Such children were substantially more likely to have a written statement related to cognitive and learning needs: 41% vs. 4%, Odds ratio = 16.9 (95% confidence interval 11.6–24.3).

Analyses were conducted in SPSS version 13.0 (SPSS Inc., Chicago).

Results

Unidimensionality and internal validity of the SAS

In order to quantify the degree to which the items measured a single latent variable, principal components analysis (PCA) was used to extract factors from the subset of questionnaires in which all ten items had been completed (N = 7,504). Only one factor had an Eigenvalue over 1: this factor had an Eigenvalue of 4.8 and accounted for 48% of variance; potential second and third factors had Eigenvalues of 0.8 and 0.7 and were not included. As shown in Table 1, all 10 items had factor loadings over 0.6 on the single factor. Cronbach’s alpha was 0.88. All individual items correlated strongly with total score: the lowest value of Spearman’s rho for any individual item was 0.60 (P < 0.001), and the highest value was 0.75 (P < 0.001). The mean score was 24.6, with a standard deviation of 6.3. The score was roughly a quarter of a standard deviation higher in females (mean 25.3, SD 6.1) than males (mean 23.8, SD 6.4); t = 10.7, 7766df, P < 0.001.

Score distribution by age

The distribution of total SAS scores is shown in Fig. 1 for three age bands: 5–8 years old, 9–12 years old and 13–16 years old. From the plot, it can be observed that the scores for each age band are slightly negatively skewed. The modal SAS score for each age band is around 20, representing a mean item score of 2, “about average”, but parents are more likely to rate their child as being “a bit better than average” or “a lot better than average” than “a bit worse than average” or “a lot worse than average”. This positive construal was slightly more pronounced with increasing age (even though parents were asked to compare their children with others of the same age). Nevertheless, the distribution does not vary greatly by age, with the curves for different ages being very similar for the low scores that are particularly relevant to screening for clinical conditions. For ease of interpretation, therefore, we present the remaining analyses in terms of raw scores without age adjustment. Almost identical results (not presented here) were obtained repeating the analyses using age-normalized T scores (mean of 50, standard deviation of 10) obtained by ranking the present data after re-weighting to be representative of the UK population (conversion table displayed on http://www.dawba.com/SAS).

Fig. 1
figure 1

Distribution of SAS score by age band

Full size image

Overlap of SAS and SDQ

Table 2 presents the correlation of SAS score with 21 different SDQ scores, reflecting three classes of informants for each of seven possible SDQ scores. The SAS-SDQ correlations were higher for the parent-based SDQ scores than for the corresponding teacher-based and youth-based SDQ scores—not surprising given that the SAS was completed by parents (“shared rater effect”). Equally unsurprising was the finding that the SAS score (higher scores are better) correlated positively with the SDQ pro-social score (higher scores are better) and negatively with all other SDQ scores (lower scores are better). It is noteworthy that none of the correlations was very high.

Table 2 Spearman correlations between the SAS score (parent-reported) and SDQ scores based on parent, teacher and youth report
Full size table

Discriminating between different disorders

Figure 2 includes receiver operator characteristic (ROC) curves showing how well the SAS score and the SDQ total difficulties score detect each condition. The better the discrimination, the closer the curve approaches the top left-hand corner, and the closer the area under the curve (AUC) approaches unity. As a guide to interpretation, an AUC of 0.5 reflects a measure that is no better than chance at discriminating between children with and without the relevant diagnosis, while an AUC of 1 would reflect perfect discrimination. Allowing for the fact that both the SAS and SDQ scores were obtained on the same sample [13], the SAS score was significantly better than the SDQ total score at discriminating between children with and without ASDs (AUC  = 0.98 for SAS; AUC  = 0.94 for SDQ; z = 3.01, P < 0.01 for the difference). Conversely, the SDQ was a more discriminating measure than the SAS for the other three disorders: hyperkinesis (AUC  = 0.89 for SAS; AUC  = 0.94 for SDQ; z = 3.019, P < 0.01 for the difference), behavioural disorders (AUC  = 0.79 for SAS; AUC  = 0.88 for SDQ; z = 7.291, P < 0.001 for the difference) and emotional disorders (AUC  = 0.65 for SAS; AUC  = 0.84 for SDQ; z = 10.967, P < 0.001 for the difference).

Fig. 2
figure 2

ROC curves showing the discriminatory power of the SAS and SDQ total difficulties score (parent rating) for four diagnostic groupings

Full size image

Figure 3 shows that the likelihood of an ASD is non-linearly related to the SAS score, with the prevalence of an ASD varying between zero and 0.6% for SAS scores over 16, but rising rapidly as the SAS score falls further. Of the 61 individuals with ASDs and a valid SAS score, 57 (93%) had SAS scores of 16 or less, 46 (75%) had SAS scores of 11 or less, and 32 (53%) had SAS scores of 5 or less. No single cut-point is ideal for all circumstances, since the choice of a cut-off depends on the population (e.g. community versus clinic) and the relative importance assigned to false positive and false negatives. In this epidemiological sample, the screening properties for three potentially useful cut-offs are as follows: with a cut-off of 16 or less: sensitivity = 0.936, specificity = 0.934, positive predictive value (PPV) = 0.104 and negative predictive value (NPV) = 0.999; with a cut-off of 11 or less: sensitivity = 0.754, specificity = 0.981, PPV  = 0.235 and NPV  = 0.998; and with a cut-off of 5 or less: sensitivity = 0.475, specificity = 0.996, PPV  = 0.525 and NPV  = 0.996.

Fig. 3
figure 3

Probability of receiving an ASD diagnosis (vertical axis) according to SAS score (horizontal axis)

Full size image

Forced-entry linear regression analyses were carried out with the SAS score as the dependent variable and generalized learning disability, ASD, hyperkinesis, behavioural disorders and emotional disorders as the independent variables. Allowing for the overlap between these conditions, the independent reduction in the SAS score (and the 95% confidence intervals) associated with each condition was as follows: a reduction of 2.1 points (95%CI 1.4–2.8) for emotional disorders, 4.1 points (3.1–5.1) for generalized learning disability, 5.9 points (5.3–6.6) for behavioural disorders, 6.9 points (5.8–8.1) for hyperkinesis and 16.9 points (15.4–18.4) for ASD.

Discussion

The SAS was designed to tap the sorts of social aptitudes that require a good ability to read social and emotional cues rapidly in complex situations in order to guide socially skilled behaviour. In a large sample of 5–16 year olds drawn from the community, the ten constituent items all loaded onto a single factor, demonstrating the scale’s internal coherence. The SAS score was roughly normally distributed, with no floor effect, but with a small ceiling effect, particularly for older adolescents.

We anticipated that low SAS scores would predict ASDs, and that less marked deficits would be associated with other psychiatric disorders or generalized learning disabilities. These effects were empirically supported. ASDs were associated with very low SAS scores—typically more than two standard deviations below the mean. By comparison, the reduction in SAS score was around one standard deviation for hyperkinesis, and less than this for other psychiatric disorders and generalized learning disabilities. These results demonstrate that the SAS score is not simply a non-specific indicator of psychopathology or learning disability.

There was clear evidence of divergent validity, with the SAS score being a more discriminating guide to the presence or absence of ASDs, and with the SDQ total score being a more discriminating guide to the presence or absence of other disorders. Many of the correlations between SAS and SDQ scores were moderate to low, providing further evidence that the two measures were not tapping identical constructs. It was striking, for example, that the parent-completed SAS score only correlated around −0.2 with the peer problems score derived from the SDQ completed by the same parent at the same assessment. This supports the conceptual distinction between social aptitude and peer relationships discussed in the introduction—good social aptitudes do not guarantee good peer relationships, and neither do poor social aptitudes preclude good peer relationships.

The SAS could potentially be used to screen for ASDs, whether in the community or specialist clinics. Its screening properties are good: the area under the ROC curve was 0.98 (where a perfect screen would have a value of 1); and at a cut-off of 16 or less, sensitivity was 0.94 and specificity was 0.93. While these results seem to compare favourably with those reported for other questionnaire measures covering social communication competence or autistic symptoms [5, 7, 16], it is potentially misleading to compare screening properties based on different sorts of samples, and judged against different “gold standard” measures. Direct head-to-head comparisons of different screening measures are more instructive [6], and we would welcome the inclusion of the SAS in any future head-to-head comparisons. It will also be important to compare the acceptability of the SAS and other possible screening measures. We predict that the brevity of the SAS and its focus on competences will make it particularly acceptable for general population screening; other screening questionnaires are longer and generally focus on symptoms and deficits.

When it comes to detecting a relatively rare disorder in the general population, even measures with apparently impressive sensitivities and specificities have unimpressive positive predictive values. Only 10% of those individuals in our sample who scored 16 or less on the SAS had an ASD (the cut-off that detected 94% of those with an ASD). Lower cut-offs can improve positive predictive value, but at the expense of poorer sensitivity. For example, with a cut-off of 5, the positive predictive value rose to 52%, but this only detected 48% of those with an ASD. Community screening programs will generally settle for fewer false negatives (missed cases of ASD) at the expense of considerably more false positives (“false alarms”), provided these false alarms can be resolved rapidly by more detailed assessment without generating a lot of parental anxiety.

Indeed, the very notion of a false alarm may be misleading since it assumes that the only purpose of screening with the SAS is to detect ASDs. An alternative perspective is that the SAS provides “wide-angle” screening that aims to detect ASDs in the course of identifying a broader group of children with low social aptitudes. With this broader group in mind, professionals reviewing children with low scores on the SAS should consider a wide range of explanations: an ASD is one possible explanation, but there are many others, including the consequences of other psychiatric disorders, a learning disability, or inadequate opportunities to learn social skills. It is not a false alarm if the professional rules out an ASD but confirms a deficit in social skills, identifies a likely cause, and advises the family or school on suitable skill-building approaches. From this perspective, the SAS could appropriately be used as an initial screening measure for general community samples, being followed up in some instances either by more definitive interviews or by a second screening phase using autism-specific questionnaires.

The study has significant limitations. Though the sample was drawn from the community, only two-thirds of those originally selected took part, which may have reduced representativeness. The SAS was only completed by parents—time constraints ruled out administering it to teachers as well, though their ratings would have been of great interest, and might potentially form a better basis for universal screening. There was no assessment of the test-retest reliability or inter-rater reliability of the SAS. The setting (a large nationwide survey carried out by lay interviewers) precluded validating the SAS by looking for correlations with direct observations of social behaviour, or with objective tests of Theory of Mind and the recognition of social and emotional cues. Similarly, the diagnosis of ASDs relied on informant reports but not direct observation. In the absence of IQ measures, the presence of generalized learning disabilities had to be judged from parent and teacher estimates of mental age. Finally, as already noted, the SAS was not compared to other possible screening measures. Redressing the limitations in this initial validation are important goals for the future.

In conclusion, the initial findings on the SAS suggest that this short measure of social ability is promising—it is certainly more than a non-specific measure of general psychopathology or learning disability. Children with low SAS scores are at an increased risk of mental health problems, with ASDs becoming particularly likely at very low scores.