Introduction

There is a critical need to identify children with autism at a very young age so that they can access evidence-based intervention that can significantly improve their outcomes (Ventola et al. 2006) and that financially benefits society by reducing the need for costly services later in life (Jacobson et al. 1998). To make early identification easier to achieve, valid screening and diagnostic instruments are needed that are, ideally, simple and brief, available at no charge, cost effective to administer, accurate when completed by both clinicians and parents, appropriate for toddlers as well as older children, and designed for individuals at both the low and high ends of the autism spectrum.

The purpose of our study was to compare diagnostic agreement, reliability, and validity for three autism instruments that have some or most of these characteristics. The instruments were the Checklist for Autism Spectrum Disorder, Childhood Autism Rating Scale (CARS), and Gilliam Asperger’s Disorder Scale (GADS). The Checklist was chosen for the study because it is the only checklist or rating scale designed to evaluate children with both low functioning autism (LFA) and high functioning autism or Asperger’s disorder (HFA). The CARS was selected because, compared with other autism rating scales, it has the best psychometric support for children with LFA (Eaves and Milner 1993; Morgan 1988; Rellini et al. 2004). The GADS was used because it is the only scale for children at the high functioning end of the spectrum appropriate for our study’s age range (early preschool through adolescence). The three instruments were completed by clinicians and parents for children with autism spectrum disorder, attention deficit hyperactivity disorder (ADHD), and typical development.

Checklist for Autism Spectrum Disorder

The Checklist for Autism Spectrum Disorder (see “Appendix” or http://psychiatry.hmc.psu.edu) is completed by the clinician based on a 15- to 20-min structured interview with the parent, information from the child’s teacher or child care provider, observations of the child, previous evaluations, and other available records. Information from the teacher can be obtained by having the teacher independently complete the Checklist prior to the parent interview. If used to diagnose autism, the Checklist should be administered by a qualified psychologist or physician specialist (child psychiatrist, developmental pediatrician, or pediatric neurologist) who has expertise and experience with autism. During the parent interview, the clinician specifically asks the parent if each of the 30 symptoms (as defined on the Checklist itself) was ever present (either in the past or currently) or absent. Symptoms 1–30 are considered present if any sub-item under the symptom is checked. Scores of 15 or higher are in the autism range. Items are counted as present on the Checklist even if they were only evident in the past. This is important because, when conducting a structured interview with a parent, counting symptoms of autism that were ever present yields higher diagnostic validity than counting symptoms that are only currently present (Lord et al. 1997). Further, another study showed that symptoms during the preschool years were more accurate than school age symptoms in differentiating children with autism from those with a language disorder (Mildenberger et al. 2001).

In a study of 143 children with autism, all children had 15 or more of the 30 Checklist symptoms (Mayes and Calhoun 1999). Another study showed that all children with clinical diagnoses of autism or Asperger’s disorder had Checklist scores of 15 or higher, and all typical children had <15 symptoms (Tryon et al. 2006). Children with LFA and HFA both score in the autism range on the Checklist (Mayes and Calhoun 2004). Norms are currently available for 630 children with autism 1–16 years of age with IQs ranging from 9 to 146. The mean Checklist score is 22, with a range of 15–30 and a standard deviation of 3. Diagnostic agreement was 100% between a psychologist using the Checklist (with a cutoff score of 15) and a child psychiatrist using DSM-IV criteria for 157 children with autism (Mayes et al. 2001).

The Checklist was developed to provide a comprehensive list of all core and associated symptoms of autism in order to help educate families and professionals regarding autism and all of its comorbid features. As noted, the Checklist is completed using parent and teacher report, clinical observations of the child, and early history of autistic symptoms. A diagnosis of autism should not be based solely on observations of the child because some symptoms occur at a low frequency and may not be observed and because some symptoms are more apparent during early childhood than later, making a parent interview a critical component of the diagnosis (Ozonoff et al. 2005).

Another goal in designing the Checklist was to create a single measure that applies to children with both LFA and HFA. Studies show that most, if not all, children with a clinical diagnosis of Asperger’s disorder actually meet DSM-IV criteria for autism (Eisenmajer et al. 1996; Howlin 2003; Manjiviona and Prior 1995; Mayes et al. 2001; Miller and Ozonoff 1997; Szatmari et al. 1995; Tryon et al. 2006). Research attempting to differentiate autism and Asperger’s disorder suggests these subgroups differ only in symptom severity or IQ (Miller and Ozonoff 2000; Myhr 1998; Ozonoff et al. 2000), indicating they are not separate disorders. Further, diagnostic agreement for autism spectrum disorder is excellent, whereas agreement for the DSM-IV pervasive developmental disorder subtypes is far lower (Mahoney et al. 1998; Stone et al. 1999; Volkmar et al. 1994). Many clinicians and researchers now concur that autism is a spectrum disorder and that Asperger’s disorder is not a separate and distinct disorder, but is at the high functioning or mild end of the autism continuum (Eisenmajer et al. 1996; Frith 2004; Macintosh and Dissanayake 2004; Manjiviona and Prior 1995; Mayes and Calhoun 2003; Mayes et al. 2001; Miller and Ozonoff 2000; Myhr 1998; Ozonoff et al. 2000; Prior et al. 1998; Schopler 1996, 1998; Wing 1998).

Childhood Autism Rating Scale

The Childhood Autism Rating Scale (CARS; Schopler et al. 1986) consists of 15 items rated on a 7-pt scale from normal to severely abnormal. Raters can include “physicians, special educators, school psychologists, speech pathologists, and audiologists” who are “trained through brief written and/or videotaped instruction to administer the CARS” (Schopler et al. 1986, p. 6). Raters base their ratings on observations, parent report, and relevant medical records (Schopler et al. 1986, p. v). The purpose of the CARS is “to identify children with autism and to distinguish them from developmentally handicapped children without autism” (Schopler et al. 1986, p. 1). In the standardization sample of 1,606 children with autism, 71% had IQs below 70 and 17% had IQs from 70 through 84. The CARS is appropriate for children of all ages, including preschoolers. The range of scores on the CARS is 15–60. Scores of 30 or higher are in the autism range. Percent agreement between the CARS and clinical diagnoses was 87% in the normative sample.

Independent psychometric support for the CARS is excellent. According to Ozonoff et al. (2005), studies report high criterion-related validity, interrater and test–retest reliability, and internal consistency, even when the CARS is completed by raters with little knowledge about autism or training on the CARS. Studies show that the CARS has better diagnostic validity than other autism rating scales (Eaves and Milner 1993; Rellini et al. 2004). CARS classification accuracy for children with autism (2–22 years of age) was 98% in a study by Eaves and Milner (1993) and 92% in a study by Sevin et al. (1991). In samples of children with autism and other disorders, agreement between the CARS and DSM-IV diagnoses was 100% in one study (Rellini et al. 2004) and 88% in another study (Perry et al. 2005). In studies of individuals with suspected autism, diagnostic agreement between the CARS and the Autism Diagnostic Interview-Revised (ADI-R) was 86% (Pilowsky et al. 1998) and 67% (Saemundsen et al. 2003). A study of toddlers who failed the Modified Checklist for Autism in Toddlers (M-CHAT), showed that diagnoses using the CARS, Autism Diagnostic Observation Schedule-Generic (ADOS-G), and DSM-IV clinical judgment agreed with each other, but not with the ADI-R (Ventola et al. 2006). Teal and Wiebe (1986) demonstrated that CARS scores significantly differentiated children with autism from children with mental retardation and no autism. However, research shows a significant negative correlation between CARS scores and both IQ and mental age (Perry et al. 2005; Pilowsky et al. 1998).

Gilliam Asperger’s Disorder Scale

The Gilliam Asperger’s Disorder Scale (GADS) is a 32-item instrument rated on a 4-point scale (from never to frequently observed) for individuals three through 22 years of age. The GADS is completed by someone who “has had regular and sustained contact with the subject for at least two weeks” including “parents, classroom teachers, educational diagnosticians, psychological associates, psychologists, and others who are familiar with the instrument” (Gilliam 2001, p. 9). The second printing of the GADS includes the Parent Interview Form, containing questions about the child’s language and cognitive development. As stated in the GADS manual, “the child should have at least average cognitive and language development” for a diagnosis of Asperger’s disorder (Gilliam 2001, p. 4). The GADS was designed to help professionals “differentiate those persons who are likely to have Asperger’s Disorder from those who do not” (Gilliam 2001, p. 6). The author acknowledges that there is no single agreed upon definition of Asperger’s disorder, that the validity of Asperger’s disorder as distinct from other pervasive developmental disorder subtypes is controversial, and that “confusion exists in differentiating persons with Asperger’s disorder from persons with autism who have higher levels of intelligence” (Gilliam 2001, p. 1).

The 32-item GADS is divided into four subscales, yielding standard scores that are based on ratings for children with diagnoses of Asperger’s disorder in the normative sample. The total of the four subscale standard scores is converted into an Asperger’s quotient using a table in the manual. The manual states “If the subject’s Asperger’s Quotient is 80 or above, the person probably has Asperger’s Disorder” (Gilliam 2001, p. 18). The GADS normative sample consisted of 371 individuals with Asperger’s disorder. Scales were completed by 253 parents, 59 educational personnel, 18 relatives, 10 psychologists, 7 speech clinicians, and 24 other raters. IQ scores were available for only 33 of the individuals. Parents, teachers, and other personnel in school districts and treatment centers were contacted and asked to complete the GADS on individuals “diagnosed as having Asperger’s Disorder” (Gilliam 2001, p. 21). Diagnoses were not independently confirmed.

The GADS manual reports adequate internal consistency for the four subscales (coefficient alphas ≥ .70). Median correlations between individual subscale items and the subscale total were .56 to .68. The manual reports an interrater reliability coefficient of .89 for the Asperger’s quotient and a test–retest reliability coefficient of .93 for the Asperger’s quotient for 10 individuals rated by teachers 2 weeks apart. Asperger’s quotients were significantly higher for 371 individuals with diagnoses of Asperger’s disorder (M 101) than for 50 individuals with autistic disorder (M 72), 28 with other diagnoses including “attention deficit hyperactivity disorder, mental retardation, emotional disturbance, and learning disabilities” (M 77), and 26 “nondisabled” individuals (M 67) (Gilliam 2001, p. 28). Using the Asperger’s disorder quotient and the four subscale scores, discriminant analysis classified individuals with and without Asperger’s disorder with 83% accuracy. A computer literature search failed to disclose any published studies reporting validity and reliability data for the GADS. A review article (Campbell 2005) stated that published scales for Aperger’s disorder, including the GADS, have significant psychometric weaknesses.

Purpose

The purpose of our study was to compare diagnostic agreement, reliability, and validity for the Checklist for Autism Spectrum Disorder, Childhood Autism Rating Scale (CARS), and Gilliam Asperger’s Disorder Scale (GADS) completed by clinicians and parents for children with autism spectrum disorder, attention deficit hyperactivity disorder (ADHD), and typical development. We selected ADHD as a clinical comparison group because ADHD is a common childhood disorder that, together with autism, comprises a large portion of referrals for child diagnostic evaluations. Children with ADHD and autism have many overlapping features, including inattention, overactivity, mood and behavior problems, early language delay, and difficulty with social skills (de Boo and Prins 2007; Mayes and Calhoun 1999, 2007; Miniscalco et al. 2007; Van der Oord et al. 2005). These shared symptoms can complicate a differential diagnosis. Therefore, determining if Checklist, CARS, and GADS scores can differentiate between HFA, LFA, and ADHD is of clinical value.

Methods

Sample

Our sample comprised 190 children with diagnoses of autism and full scale IQs below 80 (low functioning autism), 190 children with diagnoses of autism or Asperger’s disorder and full scale IQs of 80 or above (high functioning autism), 76 children with ADHD, and 64 typical children. The most common IQ tests administered were the WISC-IV, WPPSI-III, and Bayley Mental Scale. Children with autism or ADHD were patients evaluated in our diagnostic clinics by licensed PhD psychologists, board certified child psychiatrists, or a board certified developmental pediatrician using DSM-IV criteria. Typical children were children without a diagnosis of autism or other neuropsychiatric disorder recruited from colleagues and daycare centers. Sixty-one percent of the children with LFA, HFA, and ADHD had an independent confirmatory diagnosis. Comparisons of Checklist, CARS, and GADS clinician and parent scores for children with and without independent diagnoses yielded nonsignificant differences (p > .01) for 16 of the 18 scores. CARS clinician scores for children with LFA and Checklist parent scores for children with HFA were higher for children with versus without an independent diagnosis. Demographic data for the sample are presented in Table 1. Children with ADHD were older than children with autism (t = 4.4, p < .0001), and children with autism had lower IQs than children with ADHD (t = 6.9, p < .0001). IQs were not available for the typical children. Comparisons of frequencies showed significant differences for parent occupation (χ2 = 77.8, p < .0001) and gender (χ2 = 64.0, p < .0001) and a nonsignificant difference for race (χ2 = 5.9, p > .05).

Table 1 Demographic data for children (1–16 years) with low functioning autism (LFA), high functioning autism or Asperger’s disorder (HFA), attention deficit hyperactivity disorder (ADHD), and typical development (n = 520)
Full size table

Instruments and Procedure

For the children with autism or ADHD (n = 456), the Checklist, CARS, and GADS were completed by clinicians who conducted the diagnostic evaluations or provided ongoing services to the children. All clinicians had extensive experience and expertise with autism and ADHD. Clinicians included two licensed PhD psychologists, three board certified child psychiatrists, one board certified developmental pediatrician, and one certified school psychologist working at a school for children with autism. Clinicians used information from a variety of sources to complete the instruments, including an interview with the parent, observations of the child, teacher report, and a review of records. A subset of parents of the clinical children and the parents of typical children (n = 229) independently completed the same instruments for their children. Parents of the clinical children completed the instruments prior to the interview with and diagnostic feedback from the clinician. Agreement between two clinicians calculating scores on the completed parent and clinician Checklist, CARS, and GADS was 100% for a random subset of 10 children.

Data Analyses

Overall classification accuracy (percentage of children with and without autism correctly identified), sensitivity (number of children with autism scoring in the autistic range divided by the total number of children with autism), specificity (number of children without autism scoring in the nonautistic range divided by the total number of children without autism), positive predictive power (number of children with autism scoring in the autistic range divided by the total number of children scoring in the autistic range), and negative predictive power (number of children without autism scoring in the nonautistic range divided by the total number of children scoring in the nonautistic range) were calculated to assess the validity of the instruments in identifying children with and without clinical diagnoses of autism. Pearson correlation coefficients, the effect size statistic r 2 (explained variance), and percent agreement were used to indicate the degree of relationship between scores. Independent t-tests, Ancova (covarying age, gender, and parent occupation), and Cohen’s d effect size were calculated to determine the statistical and clinical significance of differences in scores between diagnostic groups. A Bonferroni correction was used when multiple comparisons were made. Clinician and parent scores were compared using dependent t-tests. The significance of differences in variable frequencies between groups was calculated using chi-square or Fisher’s exact test.

Results

Classification Accuracy

The percentages of children scoring at or above the autism cutoffs on the Checklist, CARS, and GADS are presented in Table 2. Using clinician scores and the recommended autism cutoff for each instrument, 100% of children with LFA were correctly identified using the Checklist and 97% were identified with the CARS. However, 88% of the children with LFA scored in the Asperger’s disorder range on the GADS. Among children with HFA, 99% were correctly identified using the Checklist and 75% were identified using the CARS. For the GADS, 92% of the children with HFA scored at or above the cutoff. For children with ADHD, none were identified as having autism with the Checklist or CARS and only 4% were misclassified with the GADS. As shown in Table 2, diagnostic validity using the parent scores was not quite as high but was still very positive, and none of the typical children scored in the autistic range on any of the instruments completed by parents. The percentages of children scoring at or above the autism cutoff on the three instruments did not differ significantly between clinicians and parents for LFA (χ2 or Fisher’s p > .01) but did for HFA (χ2 or Fisher’s p < .01).

Table 2 Percent of children scoring at or above the autism cutoff on the Checklist for Autism Spectrum Disorder, Childhood Autism Rating Scale (CARS), and Gilliam Asperger’s Disorder Scale (GADS)
Full size table

Table 3 presents classification accuracy percentages for children with autism or ADHD. Statistics for the Checklist included children with LFA and HFA because the Checklist is for both groups. Statistics for the CARS were only for children with LFA, because the CARS was not standardized on children with HFA. Similarly, statistics for the GADS were only for children with HFA, because the GADS was not designed for LFA. Overall classification accuracy for parents was lower than for clinicians, but still high. None of the typical children had parent Checklist, CARS, or GADS scores in the autism range. For each instrument, sensitivity, specificity, positive predictive power, and negative predictive power percentages were similar to each other, indicating that none of the instruments disproportionately over- or under-identified children.

Table 3 Classification accuracy percentages for clinician and parent scores for children with autism or ADHD
Full size table

Agreement Between Instruments

Percent agreement between the Checklist and CARS for children with LFA and between the Checklist and GADS for children with HFA is reported in Table 4. For the sample with LFA or ADHD, there was 98% diagnostic agreement between clinicians using the Checklist and CARS. For the sample with HFA or ADHD, percent agreement was 94% between the Checklist and GADS clinician scores. For children with LFA, HFA, or ADHD, correlations between clinician scores on the three instruments were highly significant (.70 for the CARS and GADS, .81 for the Checklist and GADS, and .82 for the Checklist and CARS, p < .0001, explained variance >48%). Percent agreement between instruments for the parent scores was lower than for clinician scores, particularly between the Checklist and GADS for children with HFA or ADHD. This occurred because parents tended to over-rate their children with ADHD and under-rate their children with HFA on the GADS compared with the other instruments (as indicated by the mean scores in Table 5).

Table 4 Percent agreement between instruments on diagnosis of autism versus no autism
Full size table
Table 5 Clinician and parent Scores on the Checklist, CARS, and GADS
Full size table

Differences in Scores between Diagnostic Groups

For all three instruments (Table 5), clinicians scored children with LFA and HFA significantly higher than children with ADHD (t > 20.4, p < .0001, d > 3.1). Parent scores were also higher for children with LFA and HFA than for children with ADHD (t > 6.3, p < .0001, d > 1.3), and parent scores were higher for children with ADHD than for children with typical development (t > 5.8, p < .0001, d > 1.5). On the Checklist, clinician and parent scores were approximately half a standard deviation higher for children with LFA than for children with HFA (t = 4.4 and 3.0, p < .02, d = .5 and .6). Differences between children with LFA and HFA exceeded one standard deviation for the CARS (t = 16.4 and 6.7, p < .0001, d = 1.6 and 1.1). For children with autism, the correlation between IQ and Checklist clinician scores was −.26 (explained variance 7%) and the correlation between IQ and CARS clinician scores was −.73 (explained variance 54%). Differences between the LFA and HFA groups for the Checklist and CARS clinician scores were nonsignificant when IQ was covaried (t < 1.6, p > .33). Clinician and parent scores on the GADS did not differ significantly between children with LFA and HFA, regardless of whether or not IQ was covaried (t < 1.7, p > .54, d < .4).

Clinician and Parent Agreement

Clinician and parent interrater reliability data for the 90 children with autism are presented in Table 6. Reliability coefficients for the Checklist (r = .72), CARS (r = .78), and GADS (r = .53) were highly significant (p < .0001, explained variance >28%). Clinician and parent Checklist scores did not differ significantly from each other (t = 1.6, p = .12), whereas clinicians scored children with autism higher than parents on the CARS and on the GADS (t = 3.2 and 4.6, p < .003). For the sample of children with autism or ADHD, correlations between clinician and parent scores were very significant (p < .0001, explained variance >57%) for the Checklist (r = .93), CARS (r = .87), and GADS (r = .76). Percent agreement between clinicians and parents as to whether these children fell below versus at or above the cutoff for each instrument was also high (90% Checklist, 90% CARS, and 84% GADS).

Table 6 Relationship between clinician and parent scores for children with autism
Full size table

Discussion

For the most part, diagnostic agreement, criterion-related validity, and clinician–parent reliability were very positive for the Checklist, CARS, and GADS completed by clinicians and parents. The instruments were successful in identifying children with LFA or HFA and ruling out autism in children with ADHD and typical development. These positive findings were the case even though the instruments were not always administered according to the standardized procedures described in the manuals. For our study, clinicians and parents were simply asked to complete the instruments without training or instructions beyond those included on the instruments themselves, as is often the case in clinical practice.

Checklist for Autism Spectrum Disorder

Results support the validity of the Checklist in identifying children diagnosed with LFA and HFA and differentiating these children from those with ADHD and from typical children. All of the children with LFA and 99% of those with HFA were correctly identified with the Checklist completed by clinicians, and none of the children with ADHD were misdiagnosed with autism. Diagnostic agreement between the Checklist and CARS and GADS clinician scores was very high (98 and 94%). An advantage of the Checklist compared to other instruments is that the Checklist is for all children on the autism spectrum, whereas other checklists and rating scales are for either LFA or HFA, not both. Further, the Checklist is completed using parent and teacher report, clinical observations of the child, and early history of autistic symptoms. The latter is critical in making a diagnosis because some autistic symptoms are more apparent during early childhood than later, diagnostic accuracy based on preschool symptoms is higher than that based on school age symptoms, and some symptoms occur at a low frequency and may not be directly observed by clinicians (Mildenberger et al. 2001; Ozonoff et al. 2005).

Childhood Autism Rating Scale

Our study findings are consistent with previous research demonstrating strong psychometric support for the CARS, especially for children with LFA. The CARS (which was standardized primarily on children with autism and mental retardation) completed by clinicians correctly identified 97% of the children with LFA and did not misclassify any of the children with ADHD as having autism. For children with LFA or ADHD, agreement between the CARS clinician scores and the child’s diagnosis was 98%, consistent with percentages of 88% (Perry et al. 2005), 92% (Sevin et al. 1991), 98% (Eaves and Milner 1993), and 100% (Rellini et al. 2004) reported in other studies. In our study, diagnostic agreement between CARS and Checklist clinician scores for children with LFA or ADHD was 98%, which is higher than the 86% agreement between the CARS and the ADI-R in a study by Pilowsky et al. (1998).

Our study indicated that the CARS was better at identifying LFA than HFA. For 75% of our children with HFA, CARS clinician scores were in the autism range, versus 97% of children with LFA. For children with autism, the correlation between CARS clinician scores and IQ was −.73. Other studies have also reported a significant negative correlation between CARS scores and IQ or mental age (Perry et al. 2005; Pilowsky et al. 1998). This is not unexpected because the CARS contains some developmental items (e.g., level of intelligence, language skills, and imitation ability) so that children with LFA are more likely to earn scores in the autism range than are children with HFA.

Gilliam Asperger’s Disorder Scale

A computer literature search failed to disclose any published studies reporting validity and reliability data for the GADS. Our findings were quite positive for children with HFA. These children earned a mean Asperger’s quotient of 99 on the GADS completed by clinicians, consistent with the mean quotient of 100 for the GADS normative sample (Gilliam 2001). For our children with HFA, 92% were correctly identified by GADS clinician scores and only 4% of the children with ADHD scored in the Asperger’s disorder range. Diagnostic agreement between Checklist and GADS clinician scores was 94%, and overall classification accuracy for the GADS was 93%. The latter compares favorably with the classification accuracy of 83% reported in the GADS manual (Gilliam 2001). Our mean Asperger’s quotients based on clinician scores for children with HFA (M 99) and ADHD (M 58) and based on parent scores for typical children (M 45) show the same pattern as those reported in the GADS manual for children with Asperger’s disorder (M 101), other diagnoses including ADHD (M 77), and children without disabilities (M 67). Although the GADS was standardized on individuals 3–22 years of age, diagnostic accuracy for the 1- and 2-year-olds in our study was excellent. All eight children at this age who had HFA were correctly identified by the GADS clinician scores, and none of the eight typical children at this age earned GADS parent scores in the Asperger’s disorder range.

The GADS identified children with HFA with a high degree of accuracy in our study, but most children with LFA also scored in the Asperger’s disorder range on the GADS (i.e., 88% based on clinician ratings and 72% based on parent ratings). Similarly, the mean clinician Asperger’s quotient for the children with HFA (M 99) did not differ significantly from that for children with LFA (M 101). These findings suggest that the 32-item GADS alone may not differentiate LFA and HFA. As stated in the GADS manual, “the child should have at least average cognitive and language development” for a diagnosis of Asperger’s disorder (Gilliam 2001, p. 4). The second printing of the GADS includes the Parent Interview Form (which was not available when data for our study were collected), containing questions about the child’s language and cognitive development. This or the child’s IQ should be considered with the GADS Asperger’s quotient to help differentiate between LFA and HFA.

Clinician–Parent Agreement

Interrater agreement between clinicians and parents who independently completed the instruments was high for the sample of children with autism or ADHD. Reliability coefficients were .93 for the Checklist, .87 for the CARS, and .76 for the GADS. Percent agreement between clinicians and parents on diagnoses was also high (90% for the Checklist, 90% for the CARS, and 84% for the GADS). Overall classification accuracy using parent scores was high for children with LFA and ADHD (91% Checklist, 93% CARS) and for children with HFA or ADHD (88% Checklist, 78% GADS), but was not quite as high as that for clinician scores (100, 98, 99, and 93%, respectively). The clinicians who completed the instruments in our study had extensive experience with autism and ADHD and included licensed PhD psychologists, board certified child psychiatrists, a board certified developmental pediatrician, and a certified school psychologist working at a school for children with autism. This high level of expertise, combined with the fact that accuracy was measured by agreement with the child’s clinical diagnosis, may in part explain the higher classification accuracy of clinicians than parents. Further, neither clinicians nor parents were given specific training on how to complete the instruments. Training may be more important for parents than for the experienced clinicians, which may have negatively affected the parent scores.

Clinician and parent Checklist scores did not differ significantly from each other for children with autism, whereas clinicians scored children with autism higher than did parents on the CARS and on the GADS. The better clinician–parent congruence for the Checklist compared with the other two instruments is likely (at least in part) because the Checklist is scored by the clinician using a parent interview (in addition to clinical observations and the report of teachers or child care providers), whereas the CARS and GADS are not designed as a semi-structured interview with the parent. Overall, our clinician and parent data support the validity and reliability of the three instruments for use by both clinicians and parents and the potential clinical utility of using data from both sources.

Recommendations, Limitations, and Future Directions

Further research is needed to replicate the current findings. This is the first published study comparing the validity and reliability of these three instruments, and the Checklist and GADS have little previously published data. Therefore, recommendations are offered with caution. If one needs an instrument to evaluate children at both ends of the spectrum, the Checklist may be a good option. Data supporting the CARS for LFA and the GADS for HFA are strong, whereas the CARS tended to under-identify HFA and, on the GADS, children with LFA tended to score in the Asperger’s range. Future studies are needed to determine if the CARS and GADS cutoff scores can be modified so that these instruments effectively identify children at both the high and low ends of the spectrum. The relatively high agreement between clinician and parent scores and the adequate validity of parent scores in identifying children with and without autism suggest that these instruments may be useful as a simple parent screening measure to complete in physicians’ offices, early intervention programs, and child care settings to screen for possible autism. This is especially important because the currently recommended screening tool, the M-CHAT, is only appropriate for toddlers. An alternative is needed for children 2 years of age and older.

A limitation of our study is the absence of IQ data for the typical children, who may have above average intelligence, given our recruitment method. Future research needs to be conducted with intellectually typical children. Our clinical comparison group consisted only of children with ADHD. It remains to be determined how the three instruments fare at identifying or excluding children with other disorders (e.g., social anxiety disorder or mental retardation), as well as pervasive developmental disorder subtypes (e.g., Rett’s disorder). Further, the rating scales and checklists need to be compared with other diagnostic procedures, including structured interviews (e.g., using the ADI-R) and standardized observation measures, such as the ADOS-G. Item and factor analysis of the three instruments is also needed, particularly for the Checklist, which has items and sub-items. It would be important to know if some items are redundant, so the instruments can be shortened, and if the composite score on the Checklist should include the sub-item total. Last, measures are very much needed to evaluate autism in adults. Therefore, determining the validity of the three instruments for adults on the spectrum is another area to investigate.