Introduction

Autism spectrum disorder (ASD) and intellectual disability (ID) are the most common childhood neurodevelopmental disorders, and are difficult to differentiate. According to the Diagnostic and Statistical Manual of Mental Disorders 5 (DSM-5), ASD is now considered to be a spectrum disorder characterized by impairments in social communication and interaction, restricted and repetitive behaviors (RRBs), and narrow interests [1]. According to the latest studies, the prevalence has been steadily increasing, and the global prevalence is estimated to be 0.1%–3% [2,3,4]. Patients with ID share the features of cognitive defect, adaptive deficits, and below average intellectual functioning (IQ < 70), affecting 1%–3% of the population [5, 6].

Despite the heterogeneity of ASD and ID, they exist together in most patients. Up to 70% of the ASD population has some level of ID [7, 8], and researchers have suggested that ~40% of ID cases fulfill the ASD diagnostic criteria [9, 10]. However, an ASD diagnosis in children with ID is often delayed or missed, and some are not even recognized until adulthood [11, 12]. Children with ASD and ID together have been described as having more severe social and communication impairments, more adaptive deficits, and more long-term challenging behaviors than those with either ASD or ID alone [13, 14]. Moreover, a lack of ASD-related intervention strongly influences the prognoses of these children [15, 16]. Thus, early screening and diagnosis for ASD in the ID population is of utmost importance.

However, the diagnosis of ASD still depends on the behaviors evaluated based on the ASD core symptoms, lacking objective biological markers. Evaluation instruments are believed to play crucial roles in screening and diagnosis [17]. The screening instruments available for 6- to 18-year-old children and adolescents include the Autism Spectrum Rating Scale (ASRS) [18], the Social Responsiveness Scale (SRS) [19], the Autism Behavior Checklist [20], the Autism Spectrum Screening Questionnaire [21], and the Social Communication Questionnaire (SCQ) [22]. The psychometric properties of these instruments in the general population have been systematically evaluated and validated in epidemiological studies for ASD screening across different cultures [2, 23], while the research regarding screening accuracy in ID population is limited and involves limited instruments [24]. The ASRS and SRS are widely used autistic assessment instruments for ASD screening in epidemiological surveys [23]. Our previous study showed that both the ASRS and SRS have excellent psychometric properties in screening for children with ASD in the general Chinese population, and that further studies are necessary to determine their suitability for children with other developmental neurological disorders, particularly ID [25].

Hence, the current study aimed to examine the psychometric properties of the ASRS and SRS when used in ID cases and to estimate and compare their screening accuracy for ASD in individuals with ID.

Materials and Methods

Participants

The study was conducted from January to July 2017, with participants drawn from three samples. The ASD group (6–18 years) was selected from members enrolled in a national epidemiological study of ASD in China, which was supported by the National Health and Family Planning Commission of the People’s Republic of China (201302002). The participants were all from special education schools and were diagnosed with ASD, according to the DSM-IV criteria, by a senior pediatric psychiatrist or neurologist. The ID group (6–18 years old) was from two famous special education schools, the Dong Li Feng Mei Health School and Qi Zhi School in Shanghai, which cater for all types of disabilities in students aged 3–18 years. A diagnosis of ID was confirmed by clinical diagnosis and Wechsler intelligence scale scores, including adaptive functioning evaluation, according to the China Disabled Persons Federation registration system. The TD group (6–18 years old) was from the same national epidemiological study from which the ASD cases came, and from communities on Gumei Street in the Minhang District, Shanghai. Parents of children who were unable to finish the ASRS and SRS were excluded.

Procedure

Ethical approval for this study was given by the Children’s Hospital of Fudan University Ethics Board ([2012] No.185). Parents of all eligible participants provided consent and were then invited to participate in the study. A booklet was distributed to the parents, including a consent form, a general information sheet, and brief instructions about the ASRS and SRS scales. Parents were required to complete the two scales at home on different days within a two-week period. Then, all materials were returned by mail, and the data were recorded by two staff members.

Instruments

ASRS

The ASRS is a relatively new autism screening tool developed by Goldstein et al. [26] in 2009 and is used in children and adolescents 2–18 years of age. Later, our team introduced and modified this scale, with approval from the Multi-Health System [18, 27, 28], and the results demonstrated that the modified Chinese version of the ASRS has excellent reliability and validity in identifying ASD cases from the general Chinese population. The modified Chinese version used in this study includes 59 items, each scored on a Likert scale ranging from “Never” (score of 0) to “Very Frequently” (score of 4), according to the frequency of the corresponding behavior. The screening scale has three subscales: Social Communication (21 items), Unusual Behavior (24 items), and Self-Regulation (14 items). These subscales were combined into a single composite score called the total score [18]. The raw scores were calculated first, according to the application principles of the ASRS scale, and then were transformed to standard scores, with a normative mean of 50 and a standard deviation of 10. The standard scores of the ASRS were used for all analyses in this study. The cut-off point of the total score in the general population is 60, and higher scores indicate more notable autistic traits and a greater possibility of ASD [18].

SRS

The SRS, a widely-used quantitative assessment instrument for ASD screening of individuals 4–18 years of age, was developed by Constantino et al. [29] in 2005. Additional research has shown that it has excellent psychometric properties across different cultures [19, 30,31,32]. Moreover, the SRS scores are highly correlated with the Autism Diagnostic Interview-Revised scores, the gold standard in ASD diagnosis [33]. The Chinese version of the SRS was established in Taiwan region and China’s mainland [34, 35], and both studies showed that its performance is excellent in differentiating ASD individuals from the general Chinese population. The parents’ version of the Chinese version of SRS was used in this study. The SRS consists of 65 items with a four-point Likert scale ranging from “not true” (score of 1) to “almost always true” (score of 4). Moreover, it is divided into 5 subscales: Social Awareness, Social Cognition, Social Communication, Social Motivation, and Autistic Mannerisms. To scientifically compare the data with those from Western countries, raw scores were used, as recommended [35]. The cut-off point for the total score is 56.5 in general Chinese children, and higher scores suggest more severe social deficits and autistic behaviors [35].

Statistical Analyses

Data were analyzed using the Stata statistical package (version 11.0; College Station, TX). The score distributions of the ASRS and the SRS were described in terms of the mean and standard deviation (SD). All tests were two-tailed, and P < 0.05 was regarded as statistically significant. Three-way analyses of variance (ANOVAs) were conducted, with group (ASD vs ID vs TD), gender (male vs female), and age as between-subject variables. Moreover, differences in the subscale scores between the ASD and ID groups were investigated using independent sample t-tests when the distribution was robustly normal or the Mann-Whitney U test when it was skewed. The association between the total scores of ASRS and SRS was measured by Pearson’s correlation coefficient. Receiver operating characteristic (ROC) analyses were conducted to evaluate and compare the overall degree of identification power in the ID population and the general population of the ASRS and SRS groups using the area under the curve (AUC) [36]. Based on the ROC analyses, the optimal cut-off points were determined by maximizing Youden’s J index (J = sensitivity + specificity − 1). Later, sensitivity, specificity, false-negative rate (FNR), false-positive rate (FPR), positive predictive value (PPV), negative predictive value (NPV), odds ratio (OR), likelihood ratio positive (LR+), and likelihood negative (LR–) were further calculated and compared for the diagnostic accuracy of the ASRS and SRS.

Results

Demographic Information

A total of 275 ASD cases, 71 ID cases, and 402 TD children were included in this study, and the mean participant ages were 11.69 ± 2.38, 12.56 ± 2.65, and 11.61 ± 1.75 years, respectively; there were slight differences between the three groups (P = 0.002). The male:female ratio was 6.85:1 in the ASD group, 1.03:1 in the ID sample, and 1:1 in the TD group, which showed a significant difference (P < 0.001). The proportion of different informants also showed a slight difference (P < 0.001); mothers were the main informants, with 68.6% in the ASD group, 60.6% in the ID group, and 60.5% in the TD group.

ASRS and SRS Scores in Children

The mean total scores of the ASRS and SRS by age, gender, and group are listed in Table 1. A three-way (Group*Gender*Age) ANOVA of the ASRS demonstrated a significant (P < 0.001) main effect between groups, while there was no significant effect on gender (P = 0.638) and age (P = 0.285). In addition, the ID group scored 17.70 points (1.75 SD) higher than the TD group but 8.21 points (0.75 SD) lower than the ASD group in total scores. A similar three-way ANOVA was calculated for the mean total scores on the SRS. There were significant effects for group (P < 0.001) and age (P = 0.008); however, there was almost no effect for gender (P = 0.575). The mean SRS total scores were 42.91 points (2.47 SD) higher in the ID group than in the TD group and 22.55 points (0.89 SD) lower in the ID group than in the ASD group.

Table 1 Total scores of ASRS and SRS by age and gender.
Full size table

The suited sample distribution of ASRS scores approached normal distribution (Fig. 1A and C), without any evidence of a bimodal distribution or evident violation of the hypothesis of a unitary dimensional construct [18]. Approximate normal distribution was also found for SRS scores (Fig. 1B and D).

Fig. 1
figure 1

Distribution of total scores of ASRS and SRS for the ASD group and the ID group . A, C The ASRS scores approached a normal distribution in ASD group and ID group, with median scores of 73 in the ASD group and 67 in the ID group. B, D The SRS scores showed approximate normal distribution in the ASD group and ID group, with median scores of 102 in the ASD group and 81 in the ID group.

Full size image

In addition, correlation analysis showed that the total scores of ASRS and SRS were closely associated for the entire sample (r = 0.8736; P < 0.001). Moreover, the correlation coefficients were 0.6831 (P < 0.001), 0.6673 (P < 0.001), and 0.6868 (P < 0.001) for the ASD, ID, and TD groups.

Screening Accuracy of the ASRS and SRS

One-way ANOVA and post hoc multiple comparisons of the three groups demonstrated that the ID group scored significantly higher than the TD group and significantly lower than the ASD group on the total scores and all subscale scores (Fig. 2). To analyze the discriminant validity in the ID population, we compared each score on the ASRS and SRS for the ASD and ID groups (Table 2). The ASD group scored significantly higher than the ID group on all ASRS and SRS subscales (P < 0.05), with Cohen’s d ranging from 0.406 to 0.807 for ASRS and 0.545 to 0.964 for SRS. For the ASRS scale, the ASD and ID groups differed most on the social communication subscale, with 9.41 points (t value = 5.828, Cohen’s d = 0.807). For the SRS scale, the autism mannerisms (t value = 6.826, Cohen’s d = 0.964) and the social communication subscales (t value = 6.293, Cohen’s d = 0.842) differed most between the two groups.

Fig. 2
figure 2

Comparison of ASRS and SRS scores between the three groups. *The scores in the three groups were significantly different at P < 0.05 (t-test two-tailed).

Full size image
Table 2 Comparison of scores on the ASRS and SRS scales in the ASD and ID groups.
Full size table

Moreover, we carried out ROC analyses to evaluate the overall discriminant power of the ASRS and SRS to identify ASD cases among ID cases (Fig. 3A) and to identify ASD cases in the general population (Fig. 3B). The ROC curves both showed a good ability to identify ASD in the TD group and fair performance in discriminating ASD and ID: AUC = 0.709 (95% CI, 0.642–0.776) for the ASRS total score, compared with AUC = 0.742 (95% CI, 0.675–0.808) for SRS. To further explore the discriminatory power in the ID population, we compared the AUCs of the ROC curves of different informants. The results showed no significant difference in the AUCs between fathers (AUC = 0.610, 95% CI, 0.458–0.762) and mothers (AUC = 0.740, 95% CI, 0.661–0.819) for the ASRS. We also found similar results for the SRS scale; the AUCs indicated similar discriminant validity for both informants (fathers: AUC = 0.640, 95% CI 0.490–0.791 vs mothers: AUC = 0.781, 95% CI 0.704–0.857).

Fig. 3
figure 3

ROC curves for total scores of ASRS and SRS in ASD versus ID (A) and ASD versus TD (B).

Full size image

Table 3 shows the suggested cut-off points for discriminating ASD from ID, based on the corresponding ROC curves and related indices. The ASRS had a higher sensitivity than the SRS (77.0% for ASRS vs 59.8% for SRS), but the specificity was the opposite (52.1% vs 77.5%). The FPRs and FNRs were relatively low, suggesting that the occurrence of Type I and Type II errors was relatively low. The NPVs were not high (<45%), suggesting that it was likely that a child scoring less than the cut-off point would be diagnosed with ASD, meaning that a child with ASD could easily have a missed diagnosis. However, the PPVs were relatively high (>80%), indicating that most positive results for both instruments were true ASD cases. The LR+ of the SRS was higher than that of the ASRS, as was the LR–.

Table 3 Comparison of the diagnostic accuracy of the ASRS and SRS between the ASD and ID groups.
Full size table

Discussion

The discrimination of ASD from ID is essential, but it is also challenging because of the complicated etiology, atypical symptoms in the early stage, overlapping symptoms, and a lack of effective biological markers [37]. Fortunately, evaluation tools can help to some degree. Hence, we assessed the autistic traits in individuals with ID and compared the screening accuracy of the ASRS and SRS.

We evaluated the overall distribution of ASRS and SRS in ASD cases, ID cases, and the TD group, and the results showed that individuals with ID had higher scores on total score and all subscale scores for both the ASRS and SRS than TD children. Moreover, when investigating the validation of the SRS in a Chinese population, Cen et al. [35] also found that children with mental retardation scored 53.97 points (3.42 SD) higher than TD children on the total SRS score. Similar results have indicated that individuals with ID who score higher than TD children can also be identified by the SCQ, another ASD screening instrument, which Sappok et al. [12] used to explore the validity of the SCQ for adults with ID. The higher scores in the ID population suggested that children with ID have more autistic traits than the general population. As reported, the morbidity of ASD in the ID population is significantly higher than that of the general population [11]. Moreover, ASD and ID may have some biochemical and molecular mechanisms in common [8]. Hence, we demonstrated that individuals with ID have clear autistic traits, which makes it challenging to identify ASD in the ID population.

Scoring analyses of the ASRS and SRS scales showed that gender and age did not have a significant effect, while group had a dramatic effect. In further analyses, we compared the screening accuracy of the ASRS and SRS between ASD and ID cases. The total scores for the ASRS and SRS in the ASD group were significantly higher than those in the ID group. Taking effect size (Cohen’s d) into consideration, the SRS performed slightly better than the ASRS. Second, the ASRS and SRS showed good discriminant validity in the subscale scores. That is, the scores in the two groups differed most in the social communication subscale for the ASRS and the autism mannerisms and social communication subscales for the SRS. These findings suggest that a social communication deficit is the primary characteristic of ASD that can be used to identify ASD in ID. The social communication skills, matched with the intelligence functioning in children with ID, can be lower than the intelligence levels in individuals with ASD [38]. Another pivotal characteristic of ASD, RRBs, were evaluated in the unusual behaviors of the ASRS, which is defined as autistic mannerisms of the SRS, and the results also showed significant differences between the ASD and ID groups. The frequency and severity of RRBs have been associated with the severity of ASD in patients [37, 39].

The ASRS and SRS have shown excellent psychometric properties in screening for children with ASD among the general population [25], as they did in our study; however, the screening accuracy of these two scales in individuals with ID had not been systematically evaluated. Our ROC analyses demonstrated fair performance in identifying ASD in individuals with ID, while the SRS performed slightly better, with a higher AUC, which did not show significant differences between the two scales. The results of both scales were also similar for different informants (fathers and mothers). The fair performance of the two scales might be attributed to some extent to the equivalence of the ASD group to the ASD+ID group in this study, as they were recruited from special education schools with admission requirements including a Wechsler intelligence scale score <70 or other disabilities, and the two scales may perform better in discriminating ASD alone in the ID population. Moreover, the ASRS showed higher sensitivity, and the SRS showed higher specificity. The PPVs were both high, suggesting that most positive cases screened in ASRS or SRS were true ASD cases. The LR+ of the SRS was slightly higher than that of the ASRS, indicating that the SRS had a slightly better ability to identify ASD in ID than the ASRS, as was the LR–, indicating that the SRS might be slightly more likely to misjudge an ASD case for a typical child than the ASRS.

In addition, to achieve better screening accuracy, the cut-off point should be set slightly higher than in the general population with regard to the ID population. Here, we found that 67 and 96 were the cut-off points for the ASRS and SRS, respectively, based on the ROC analyses. But they are both usually ~60 in the general population [18, 30, 34]. Cen et al. [35] have suggested a cut-off of 77.5 when the SRS is used for discriminating between ID and ASD for all levels of intellectual functioning. The sensitivity was 0.748, specificity 0.603, and AUC 0.692, and the authors also suggested a cut-off of 56.5 in the TD group. Previous studies, using another widely used ASD screening scale, the SCQ, have also recommended adjusting the cut-off point when applying it to individuals with ID [12, 24].

Above all, the two scales showed quite similar screening accuracy for identifying ASD among ID cases, and the total scores were closely associated, according to the correlation analysis. Note that ASRS is an overall assessment tool for the features that are characteristics of ASDs, including social communication, self-regulation, and unusual behavior [18]. However, the SRS is primarily focused on the multidimensional comprehensive evaluation of social skills, which occupy 53 of 65 items, and the remaining 12 items assess autistic mannerisms [35]. Combined screening with the ASRS and SRS may improve the screening accuracy in the ID population, but this combination requires further study.

To the best of our knowledge, this study is the first to focus on identifying ASD cases in a pediatric ID population, while previous studies have primarily focused on adults with ID. Second, a relatively new scale ASRS and a widely used scale SRS were used in this study, and the scores were compared later.

The first limitation of this study is that the specific IQ of an individual could have been measured at the same time and detailed comparisons in different IQ subgroups (ranging from mild to profound) could have been made for further study. Second, there may be example selection bias due to the sample size and the participants were from one city of China; a multicenter and nation-wide study with a larger sample size should be considered in the future.

In conclusion, individuals with ID have evident autistic traits, and identifying ASD conditions in an ID population is challenging. While assessment tools such as the ASRS and SRS help to some degree, their scores are highly associated and increasing the cut-off point according to the ROC is recommended when using the ASRS and SRS to identify ASD in an ID population.