Introduction

The existence of psychiatric syndromes within the autism spectrum disorder (ASD) clinical phenotype has been a matter of controversy for many years. Perhaps the most contentious element in this debate is which of its behavioral, affective and cognitive concomitants are epiphenomena of the ASD diathesis or “true” psychiatric disorders. Support for the latter was garnished from a number of case studies of individuals with ASD who appeared to meet conventional diagnostic criteria for a wide range of nonASD psychiatric syndromes (reviewed by Sverd 2003). Inspired to some extent by these reports, researchers began to evaluate larger groups of individuals with ASD using conventional psychiatric assessment instruments, and their findings have been published only within the past decade (e.g., Gadow et al. 2005; Leyfer et al. 2006; Simonoff et al. 2008; Witwer and Lecavalier 2010). It now seems clear that many if not most youth with ASD do meet traditional diagnostic criteria for at least one of the more prevalent nonASD psychiatric disorders. However, owing to the relative recentness of this emerging evidence, less is known about the interrelation of various illness parameters (e.g., impairment, number and severity of symptoms) or their implications for defining disorders (caseness) or differences between parent and teacher ratings of symptoms (informant discrepancy) compared with non ASD clinical populations.

According to the Diagnostic and Statistical Manual of Mental Disorders-4th Edition (DSM-IV-TR; American Psychiatric Association [APA], 2000), symptom-induced impairment in social, academic, or occupational functioning is an essential requirement for the diagnosis of psychiatric disorders. Impairment is generally conceptualized as negative consequences arising from the symptoms of the disorder (Bird et al. 1990), and although it overlaps with symptom severity, they are not equivalent. In fact, the available evidence for nonASD samples indicates modest association between the degree of impairment and number or severity of symptoms (reviewed by Rapee et al. 2012). The distinction between impairment and symptom severity has a number of practical and clinical implications, one of which can be summarized as follows: In everyday clinical settings, behavioral, emotional, and cognitive symptoms are typically assessed with behavior rating scales, which are a cost-effective way of obtaining information about child functioning in diverse settings. However, few broad-band DSM-IV-referenced scales assess disorder-specific impairment, and for a variety of reasons, impairment rating scales are not widely used today, and unless linked to the symptoms of specific disorders, lack precision. Therefore, severity of symptoms often becomes the primary indicator of the potential need for clinical services, especially for measures that are norm-referenced. This situation raises an important question as to whether or not children who are identified as exhibiting the prerequisite symptoms or have elevated symptom severity scores are in fact impaired. This topic has special significance for the care of children with ASD because they are at high risk for co-occurring psychiatric syndromes (e.g., Leyfer et al. 2006; Simonoff et al. 2008; Witwer and Lecavalier 2010) and consequently treatment with psychotropic medication (Mandell et al. 2012). In other words, discrepancies among parameters of mental health functioning have important practical implications for service delivery. It is also likely that for some children the core features of ASD (i.e., social and communication deficits and repetitive behavior and interests) influence the way informants perceive nonASD psychiatric symptom-induced impairment and ultimately caseness.

Children with ASD

To date, only a small handful of studies have conducted comprehensive assessments of DSM-IV-defined co-occurring psychiatric syndromes in children with ASD, and even fewer investigations have reported on caregiver perceptions of psychiatric symptom-induced impairment. In one study of 109 children and adolescents (ages 5–17 years) with ASD, Leyfer et al. (2006) differentiated between youth who met full DSM-IV diagnostic criteria and those who had a “subsyndromal diagnosis” utilizing a structured psychiatric interview developed for individuals with ASD. A subsyndromal diagnosis was defined as “a significantly impairing psychiatric syndrome… that [fell] just short of meeting DSM criteria” p. 852). Participating families were recruited from community sources and were participating in either a longitudinal or brain imaging study. They found that a large proportion of participants had subsyndromal diagnoses. For example, 31 % met full criteria for attention-deficit/hyperactivity disorder (ADHD), but an additional 25 % of the sample had subsyndromal ADHD (lifetime prevalences). Similarly, 10 % met full diagnostic criteria for at least one major depressive episode, and an additional 14 % of the sample had subsyndromal major depression. In another study, Witwer and Lecavalier (2010) also used a structured psychiatric interview to evaluate 61 children and adolescents (6–17 years) with ASD who were recruited from community sources and who had significant emotional or behavioral problems. Rates of nonASD psychiatric impairment were very high (up to 90 %) although not all children met symptom count criteria. As an example, 61 % were impaired by major depressive disorder or dysthymia symptoms, but only 31 % of the total sample met symptom count criteria for either disorder, and only 28 % met impairment plus symptom count criteria. Collectively, these findings suggest discrepancy between caregiver perceptions of symptom-induced impairment and DSM-IV criteria for caseness but leave unanswered numerous questions.

Objectives

The overall goal of the present study was to determine the association between impairment and other illness parameters such as the number and severity of DSM-IV psychiatric symptoms in children with ASD and to illustrate their clinical significance through comparison of associated prevalence rates. Specific objectives were to characterize the extent to which co-occurring psychiatric symptoms are perceived by caregivers as impairing social and academic functioning (Objective 1). This is important because perceived impairment plays a major role in help-seeking and eventual referral for educational, behavioral, and pharmacological intervention. We also describe how symptom-induced impairment as a criterion for caseness influences obtained rates of co-occurring psychiatric syndromes compared with other commonly used criteria (Objective 2). Here we were specifically interested in how prevalence rates based on an impairment cutoff agreed with rates based solely on the prerequisite number of symptoms (i.e., symptom cutoff) and how these cutoffs differed from rates based on a combination of these two criteria (i.e., clinical cutoff or impairment + symptom cutoff). Severity is another common illness parameter, but to the best of our knowledge, there are no prior studies examining the relation between impairment and symptom severity in children with ASD (Objective 3). Although there is a growing literature pertaining to informant discrepancy and severity of psychopathology in children with ASD (Gadow et al. 2006; Lecavalier et al. 2006; Szatmari et al. 1994), virtually nothing is known about agreement regarding impairment (Objective 4). This is also a major clinical concern as mental health referrals emanate from both sources, and disagreements about impairment contribute to home-school conflicts about the proper course of action.

To address these objectives, we examined diagnostic intake information obtained for 6–12 year old children with ASD who were consecutive referrals to a developmental disabilities clinic. Parents and teachers completed a well-validated DSM-IV-referenced rating scale. In order to control for the obvious confounds associated with the use of different assessment instruments to evaluate different illness parameters, the scoring algorithms for these parameters (i.e., number, severity, impairment, clinical) were derived from this one measure. Owing to the limited amount of research on this topic in children with ASD, by necessity our predicted outcomes are based in part on the extant literature with nonASD samples (reviewed by Rapee et al. 2012). Because (a) research indicates high rates of co-occurring symptoms and disorders in children with ASD; (b) a significant minority of non ASD children who are impaired by psychiatric symptoms do not meet full DSM-IV symptom cutoff criteria; and (c) parent and teachers often do not agree about the severity of psychiatric symptoms, a large percentage of children were expected to be impaired by co-occurring psychiatric symptoms, with differentially higher rates based on both informants. It was also expected that the impairment cutoff criterion would result in the highest prevalence rates, and impairment cutoff plus symptom cutoff criteria (referred to here as clinical cutoff) would be associated with the lowest rates, with symptom cutoff intermediate. We reasoned that with a highly “co-morbid” phenotype such as ASD, it would be easier for an individual to meet impairment cutoff criteria in the absence of full symptom cutoff criteria as a consequence of having multiple problems compared with a nonASD sample comprised of children with less developmentally severe neurobehavioral syndromes. Also, the requirement of meeting two criteria (i.e., impairment cutoff + symptom cutoff) would result in lower rates than each individually. We predicted relatively modest correlation between impairment and symptom severity, which would suggest the two constructs tap into different illness parameters. Finally, if one assumes that informant perceptions of impairment are less contextually bound than frequency of occurrence of specific behaviors, then it would be reasonable to predict differentially greater informant discrepancy for symptom cutoff than impairment cutoff.

Method

Participants

Case records for consecutive referrals to a university hospital developmental disabilities specialty clinic (2004–2010) were screened for children who were between 6 and 12 years old (mean = 8.5, SD = 1.8) at time of evaluation and had both a diagnosed ASD and a parent- and/or teacher-completed Child and Adolescent Symptom Inventory- 4th Edition Revised (CASI-4R; Gadow and Sprafkin 2005). The child sample (N = 115) was primarily male (86 %) and self-identified European geographic ancestry/Caucasian/White (91 %). IQ test scores were available for 95 (83 %) of the children: Mean Full Scale IQ = 85 (SD = 23; range = 40–128), with 24 (23 %) functioning in the intellectual disability range. All children met DSM-IV criteria for an ASD by best clinical estimate (see Procedure): autistic disorder (31 %), Asperger’s Disorder (19 %), and PDD-NOS (50 %). Most children were already receiving some type of intervention, including psychotropic medication (26 %) or special education (75 %). This study was approved by a university Institutional Review Board and appropriate measures were taken to protect child and caregiver confidentiality.

Measures

Mothers and teachers rated psychiatric symptoms using the Child and Adolescent Symptom Inventory-4R (CASI-4R; Gadow and Sprafkin 2005), which is designed for evaluating youth 5 to 18 years. Respondents were asked to indicate which frequency rating “best describes this youth’s overall behavior”. Individual items bear one-to-one correspondence with DSM-IV symptoms and are rated on a scale from 0 (never) to 3 (very often). Both forms cover a range of disorders, including the following which are considered in this study: ADHD inattentive subtype, ADHD hyperactive-impulsive subtype, ADHD combined subtype, oppositional defiant disorder (ODD), conduct disorder (CD), generalized anxiety disorder (GAD), social phobia, major depressive episode (MDE), dysthymia, manic episode, schizophrenia, and separation anxiety disorder. The items in parent and teacher versions are identical; however, a few symptoms that are unlikely to be observed in the school were omitted from the teacher version (CD, MDE, dysthymia), and symptoms of separation anxiety disorder appear only in the parents’ form. The last item in each symptom subscale addresses impairment by asking the informant “How often do the behaviors in [this category] interfere with youth’s ability to do schoolwork or get along with others”. For the ADHD subscale, there is one impairment item for all subtypes.

There are several scoring procedures, each of which has certain advantages for specific clinical and research applications: The Symptom Severity score is the sum of all item scores from a specific subscale, which can be converted to a T-score utilizing gender-, age-, and informant-specific norms. The symptom count score is the total number of symptoms in a specific subscale considered to be problematic (i.e., rated often or very often). Youth receive a symptom cutoff score if the informant endorses the prerequisite number of DSM-IV symptoms specified as being necessary for a clinical diagnosis. For example, in the case of ADHD, the inattentive and hyperactivity-impulsive subtypes are each defined as six or more of their respective nine symptoms being rated often or very often. To be classified as ADHD combined type, individuals must meet symptom cutoff criteria for both inattentive and hyperactive-impulsive subtypes. Therefore, the symptom cutoff scoring algorithm parses children into four mutually exclusive ADHD categories to include a no-ADHD group. Impairment severity is rated on a 4-point scale, and the impairment cutoff score is defined as a frequency rating of often or very often. A clinical cutoff score requires both a symptom cutoff score and an impairment cutoff score and therefore best approximates DSM-IV diagnostic criteria.

Numerous studies indicate the CASI-4R subscales demonstrate satisfactory psychometric properties. Specifically, individual symptom dimensions evidence satisfactory internal consistency (Cronbach’s alpha), test-retest reliability, and convergent and divergent validity with respective measures from a range of relevant assessment instruments and diagnostic procedures in community-based normative, clinic-referred nonASD, and ASD samples (see Gadow and Sprafkin 2011). Confirmatory factor analysis supports the internal validity of the DSM-IV model of behavioral syndromes in a large sample (N = 498) of children with diagnosed ASD (Lecavalier et al. 2009). The findings of numerous studies have found that CASI-4R global severity scores are minimally correlated with age, gender, IQ, and SES.

Procedure

Prior to their initial evaluation, parents of potential patients completed an intake assessment battery that included behavior rating scales, background questionnaire, and permission for release of school reports. Parents delivered a similar packet of materials to the school with instructions that requested teachers to complete several rating scales and the school to provide copies of psycho-educational evaluations and special education evaluation records. Schools mailed their information directly to the clinic. Parental ratings were completed primarily by the child's mother (>90 %). The number of children for whom parent and teacher ratings were available was as follows: both informants (n = 77), parent only (n = 27), and teacher only (n = 11).

Intake evaluations included interviews with the children and their caregivers; informal observation of parent–child interaction; and review of the assessment battery. ASD diagnoses were confirmed by an expert diagnostician and based on five sources of information: (a) comprehensive developmental history, (b) clinician interview with child and caregiver(s), (c) direct observations of the child, (d) review of validated ASD rating scales including the CASI-4R (DeVincent and Gadow 2009; Gadow et al. 2008), and (e) in most cases (75 %) the Autism Diagnostic Observation Schedule (ADOS; Lord et al. 2000) administered by a certified examiner. The only exceptions were children with a prior well-documented diagnosis of ASD (e.g., prior clinician or school evaluations) who received all of the aforementioned assessments but not the ADOS. As previously reported (Gadow and DeVincent 2012), these cases were more likely to have lower IQ scores, current special education services, and more severe ASD communication deficits than the ADOS-assessed group.

Statistical Analyses

Correlations of impairment severity scores with symptom count and symptom severity scores were conducted using Spearman’s rho. As a rule of thumb for determining the magnitude of correlations, Cohen (1988) suggested the following: r >0.50 = large, 0.50–0.30 = moderate, and 0.29–0.10 = small. Differences in the strength of association of impairment with number and severity in respective parent- versus teacher-completed measures were tested using Fisher’s r-to-z' transformation. Agreement between parents’ and teachers’ cutoff scores (above vs. below cutoff) were calculated for impairment cutoff, symptom cutoff, and clinical cutoff for each targeted condition using Cohen’s kappa. One acknowledged limitation of kappa is that it is less sensitive for estimating agreement in the case of disorders with low base rates. Agreement was also calculated within raters between impairment cutoff and symptom cutoff to determine similarities between the two non-overlapping criteria. In interpreting kappa, we used the criteria proposed by Altman (1997) and Kraemer et al. (2012), which are as follows: ≤ 0.20 ( poor), 0.21–0.40 (fair or acceptable), 0.41–0.60 (moderate or realistic), 0.61–0.80 (good), and ≥0.80 (very good).

Results

Impairment Criterion Rates (Objective 1)

Parents rated 81 % of children as meeting impairment cutoff for at least one targeted psychiatric disorder, and in rank order some of the more common conditions were ADHD (67 %), ODD (35 %), GAD (32 %), social phobia (28 %), and depression (20 %; see Table 1). The combined rate of impairment for the three anxiety disorders was 47 %. Teachers’ ratings indicated an even higher rate (86 %) of psychiatric impairment for at least one co-occurring disorder. In rank order, the most common conditions were ADHD (71 %), ODD (43 %), social phobia (38 %), GAD (34 %), and depression (28 %). The combined rate for the two anxiety disorders rated by teachers was 55 %. Of the 77 children for whom both parent and teacher ratings were available, 72 (94 %) met impairment cutoff for at least one targeted disorder.

Table 1 CASI-4R prevalence rates for screening cutoff, symptom cutoff, and clinical cutoff scores
Full size table

Agreement Between Impairment, Symptom, and Clinical Cutoff Criteria (Objective 2)

As expected, prevalence rates differed as a function of the criteria used to determine caseness. Rates based on impairment cutoff scores were higher than rates based on symptom cutoff scores, and the lowest rates were for clinical cutoff scores (Table 1). Across informants, prevalence for any disorder was high regardless of criterion used to determine caseness.

The median number of conditions for which parents and teachers rated the child as meeting symptom cutoff was one (range = 0 to 8 of 10 possible disorders for parents and 0 to 6 of 9 disorders for teachers). Of children who met symptom cutoff criteria for multiple disorders, both parents and teachers rated them as impaired for most, if not all, disorders. For example, one parent rated his or her child as meeting symptom and clinical cutoff for eight psychiatric disorders. This resulted in high agreement (kappa) between symptom cutoff and impairment cutoff, with discrepancies primarily due to meeting impairment cutoff without symptom cutoff. Agreement between these non-overlapping cutoffs for parents’/teachers’ ratings was moderate to good for several disorders including any ADHD (0.70/0.55), ODD (0.79/0.51), CD (0.71, teachers’ ratings), GAD (0.71/0.48), social phobia (0.55/0.37), separation anxiety disorder (0.58) and dysthymia (0.64/0.33). Agreement was less strong for parent-rated CD (0.27) or parents’/teachers’ ratings of MDE (0.19/0.11), manic episode (0.27/0.34) or schizophrenia (0.18/0.34).

The requirement that symptoms be impairing generally resulted in relatively little decrease in prevalence over and above rates based solely on the number of prerequisite symptoms (false positives). By requiring impairment (i.e., clinical cutoff), the median reduction in the number of cases based only on symptom cutoff was 13 % for parents’ ratings and ranged from 0 % for ADHD combined type, GAD, separation anxiety, and schizophrenia to 46 % for CD. For teachers’ ratings, the median reduction rate was 0 % (ranging from 0 % for ADHD hyperactive-impulsive type, ADHD combined type, ODD, MDE, manic episode, and schizophrenia to 24 % for GAD).

However, more children were above impairment cutoff than symptom cutoff (false negatives), and this was true for all disorders (i.e., many children whom parents and teachers rated as impaired by psychiatric symptoms did not meet DSM-IV symptom count criteria for the specified disorder). For parents’ ratings, the median reduction from impairment cutoff to clinical cutoff was 52 % (ranging from 11 % for any type of ADHD to 88 % for schizophrenia). Teacher ratings were similar, with a median reduction of 58 % (ranging from 31 % for any type of ADHD to 81 % for schizophrenia).

Association of Impairment with Number and Severity of Symptoms (Objective 3)

With few exceptions, impairment severity scores were moderately to highly correlated with both the number of symptoms rated as problematic and the severity of symptoms, and this was true for both parents’ and teachers’ ratings (Table 2). Nevertheless, there were a few notable exceptions. Teachers’ ratings were more strongly correlated for ADHD combined type (number) and CD (number, severity) symptoms than parent’s ratings. Conversely, parents’ ratings were more strongly correlated for severity of ODD, GAD, and social phobia.

Table 2 Spearman correlations of CASI-4R impairment severity scores with number and severity of symptom scores
Full size table

Agreement for Parent and Teacher Cutoffs Scores (Objective 4)

For the 77 children with both parent and teacher ratings, agreement between raters was poor to fair regardless of the type of cutoff used with the exception of conduct disorder, which was in the moderate range (Table 3). For symptom cutoff, agreement was generally comparable or lower than for impairment cutoff scores, with the notable exception of MDE. Agreement across raters was greatest for clinical cutoff scores. However, the magnitude of these agreements was poor to fair, with the exception of MDE. As was the case in prior analyses, ADHD symptom cutoff subtypes are based on mutually exclusive groups. For example, the kappa (0.27) for the ADHD inattentive subtype refers to children that parents and teachers considered had this and no other subtype of ADHD. There was one impairment item for all 18 ADHD symptoms; therefore, we only calculated agreement for the “any ADHD” impairment cutoff score because it reflects whether parents and teachers considered ADHD symptoms in their entirety to be impairing.

Table 3 Agreement between parents’ and teachers’ CASI-4R impairment, symptom, and clinical cutoff scores
Full size table

Discussion

To the best of our knowledge, this is the first study to use a broad-band DSM-referenced rating scale to examine the relation of impairment with number and severity of a wide range of psychiatric symptoms in children with ASD, teacher perceptions of impairment, convergence between parent and teacher perceptions of impairment, and rates of impairment and disorders based on a combination of parent and teacher report. Certainly one of the most thought-provoking findings from this investigation is the high rate of impairment attributed to psychiatric symptoms in the study sample (Objective 1). Parents (81 %) and teachers (86 %) rated the vast majority of children as being impaired “often” or “very often” by the symptoms of at least one targeted disorder. For purposes of comparison, a similar study conducted with 6–12 year old nonASD children referred for outpatient psychiatric evaluation (N = 284) found comparably high rates of impairment (Gadow, Kaat and Lecavalier in press). However, impairment rates are much lower in non-referred samples. For example, a study of nonASD youth from primarily economically challenging environments found that approximately 22 % were impaired by the symptoms of at least one targeted disorder according to caregiver (CASI-4R) or self-report (Gadow et al. 2012a). The most common impairing disorders for children in this study according to informants were ADHD (parents, teachers), ODD (parents, teachers), and anxiety disorder, which is also consistent for the most part with research using the CASI-4R with non ASD community-based (Gadow et al. 2012a) and clinic-referred (Gadow et al. in press) youth.

Research has clearly established high rates of co-occurring psychiatric symptoms in children with ASD, and our results for symptom cutoff scores both support this conclusion and replicate prior efforts (Gadow et al. 2005) with a new, independent sample. It is noteworthy that not only are symptom cutoff prevalence rates for each disorder similar in the two studies, but the relative distribution of disorders with regard to each other is highly similar (e.g., rates of ADHD were high whereas rates of CD are low).

In the present study, a larger percentage of children met impairment cutoff criteria for specific disorders than DSM-IV symptom cutoff criteria; nevertheless, agreement (kappa) between impairment and symptom cutoffs was moderate for most of the more common targeted disorders (Objective 2). This suggests that symptom cutoff scores provide reasonable estimates of symptom-induced impairment in 6–12 year old children with ASD (cf. Gadow et al. 2005). Although many youth in community-based or non-psychiatric samples who meet symptom cutoff criteria are not perceived as being impaired by their symptoms (e.g., Angold et al. 1999; Gadow et al. 2012a; Gathje et al. 2008; Gordon et al. 2006; Rapee et al. 2012), this appears to be much less the case in psychiatric referrals (Gadow et al. in press). In this regard, Angold et al. (1999) noted that compared with those meeting symptom count criteria alone in the Great Smoky Mountains Study, impairment was more predictive of clinical service use initially and at 1-year follow-up, which suggests that help seeking behavior is likely prompted more by perceived impairment than symptoms.

As predicted, the discrepancy between cutoff criteria was larger for impairment cutoff versus clinical cutoff scores than symptom versus clinical cutoff for some (e.g., ODD) but not all (e.g., social phobia) disorders. In other words, strict adherence to DSM-IV diagnostic criteria may miss a significant minority of children who may be in need of clinical services. This is consistent with the findings of other studies of youth with ASD, which include clinic-referred (Witwer and Lecavalier 2010) and community-based (Leyfer et al. 2006) samples, as well as research with typically-developing youth (e.g., Angold et al. 1999; Gadow et al. 2012a) and non ASD clinic referrals (Gadow et al. in press).

Association Between Impairment and Symptoms

As previously noted, many experts make a clear distinction between severity of impairment and symptoms, and we therefore predicted a modest degree of correlation between these two illness parameters (Objective 3). In the present study we obtained moderate to high correlations (≥0.60) for most disorders and for both raters between impairment and symptom severity scores, whereas studies of non ASD samples using very different assessment instruments have generally reported low correlations between these variables (cf. Rapee et al. 2012), thus supporting their independence as illness parameters. Why investigations of non ASD samples find more modest correlation between impairment and severity is a puzzle, but studies are few and methodologies diverse. As previously indicated, one plausible explanation pertains to participant selection. Youth with more severe ratings who are not perceived as being impaired are unlikely to receive psychiatric evaluations (but would nevertheless be included in epidemiologic or nonpsychiatric samples) possibly resulting in greater convergence in referred samples.

Although impairment cutoff shows acceptable agreement with symptom cutoff (Objective 2) and severity of impairment is related to symptom count and symptom severity (Objective 3), impairment cutoff rates were consistently high. One reason for this may be due to how the cutoffs are calculated. For example, DSM-IV diagnostic decision rules require the presence of one or two specific symptoms from a larger set (e.g., GAD, MDE, schizophrenia) or subsets of symptoms (e.g., ADHD combined type) for some conditions, and CASI-4R symptom cutoff scoring algorithms are consistent with these specifications. However, the CASI-4R impairment cutoff scoring algorithm does not stipulate that these required symptoms (or subset of symptoms) contribute to the youth’s perceived impairment. Therefore, it is possible to be impaired by many or most symptoms of a disorder, but not the defining ones. Conversely, CASI-4R Symptom Severity and Impairment Severity scoring rules treat all symptoms equally, and in the case of the former, resulting scores are simply the sum of individual items. For these reasons, our reported correlations between impairment and symptom severity should not be interpreted as representing associations for specific psychiatric diagnoses but rather symptom dimensions (see Sprafkin et al. 2013).

Informant Agreement

In general, agreement (kappa) between parent and teacher impairment, symptom, and clinical cutoff scores was poor to fair, and contrary to prediction, was only somewhat better for impairment cutoff versus symptom cutoff, i.e., both appear to be equally context bound (Objective 4). Therefore, the well-documented discrepancy between parents’ and teachers’ ratings of psychiatric symptoms pertains to impairment as well. Informant discrepancy has important clinical implications (Drabick and Gadow 2012; Gadow and Drabick 2012), one example of which is estimating the extent of mental health concerns. For example, all studies of DSM-IV-defined psychopathology in children with ASD either report rates based on structured interviews with the child’s parent(s), or if using multiple informants, rates for each informant separately. In the present study the prevalence of any type of ADHD based on parents’/teachers’ clinical cutoff scores were 60 % and 49 %, respectively. However, the combined rate for ADHD based on either informants was 82 %, which suggests that the sole reliance on one informant likely underestimates the true extent of impairment just as it does prevalence when using traditional diagnostic criteria.

Although research supports the distinction between impairment and symptom severity, little is known as to whether associations between these constructs exhibit informant discrepancy. In the present study, there were statistically significant informant-related differences in correlations of impairment with number and severity of symptoms. In addition to their clinical implications, these discrepancies may have practical significance for the construction of better informant-specific phenotypes. In other words, one suggestion for improving informant agreement is to make the symptoms that define clinical phenotypes more source-specific. Our findings underscore the complexity of this proposal as it will likely need to consider impairment as well.

Informant discrepancy, long considered to be a nuisance variable reflecting error variance, has emerged to become a legitimate area of study (Drabick and Gadow 2012; Gadow and Drabick 2012; De Los Reyes 2011). As with many psychosocial variables, its underlying mechanisms are multifactorial, but contextual specificity likely figures prominently in this phenomenon (Gadow and Drabick 2012). It is also possible that informants have a difficult time detangling impairment associated with the core features of ASD from other co-occurring symptoms (Witwer and Lecavalier 2010), which in turn increases the probability of disagreement. However, because ASD shares phenomenological similarities with many co-occurring psychiatric disorders (Rommelse et al. 2011), the significance of at least some distinctions may be moot.

Strengths, Limitations, and Future Directions

The present study has several strengths including a relatively large sample of youth with ASD, restricted age range, multiple informants, multiple disorder-specific illness parameters based on information from the same measure, and aggregation of prevalence data across informants; nevertheless generalization of results is subject to several qualifications. The sample is heterogeneous with regard to ASD clinical presentation and intellectual ability; nevertheless, it is representative of clinic referrals and therefore in keeping with the clinical orientation of the study’s objectives. Symptom-induced impairment was based on one global evaluation item for each disorder, but it is possible that a more broad-based strategy with more comprehensive measures will generate additional insights into this topic. For example, Bird et al. (2000) have suggested that more specific questions directly probing for negative effects related to symptoms within a diagnostic category may be more precise measures of impairment.

A related issue is the extent to which the severity of co-morbid psychiatric impairment is attributable to ASD or its associated clinical features (e.g., cognitive deficits). For example, impairment resulting from ASD may influence perceptions of non ASD psychiatric impairment or make it difficult to differentiate their independent contributions. Another important topic is trying to understand the processes that influence perceptions of impairment and subsequent help seeking behaviors in children with ASD. There are important differences between children whose symptoms are evident in multiple settings (cross-informant) versus youth whose symptoms are problematic primarily in the home or school, but not both (informant-exclusive; Gadow and Drabick 2012), and future research will need to address the implications of contextual variation for perceptions of impairment. Although IQ shows relatively low correlation with CASI-4R subscale scores, less is known about its association with impairment. In the present study, a substantial percentage of youth (23 %) had IQs < 70, but studies with larger samples will be required to address this topic in greater detail.

Some children (26 %) were receiving psychotropic medication at the time their behavioral evaluations were being conducted, and this may have influenced ratings. It is unclear, however, how this might have altered the general conclusions of the study as this clinical population appears to respond less satisfactorily to a range of psychotropic medications compared with nonASD patients (Siegel and Beaulieu 2012). It would have been ideal to obtain all assessments when children were not receiving medication, however, withdrawal from treatment solely for the purpose of obtaining ratings raises a number of concerns. Equally problematic would be the exclusion of these children, which could potentially bias the sample with regard to the very behaviors that were the object of study.

It was not the intent of the present study to determine which scoring algorithm was “best” but rather to illustrate their convergence-divergence with regard to prevalence estimates and informant discrepancy. As discussed elsewhere (Gadow et al. 2012a), each type of score has advantages and disadvantages with regard to specific research and clinical applications. For example, if referral for clinical services and eventual treatment is the desired outcome variable, then impairment scores may be the better predictors than severity or count variables. Conversely, symptom severity scores may be better predictors of stress in the home or differentially more useful in the identification of biomarkers for long-term outcome. Regardless, our results do support consideration of a wider range of illness parameters in future research.

Other topics that warrant investigation but are nevertheless beyond the scope of the present study are ways of improving agreement between specific illness parameters. For example, as previously noted, some diagnoses require the endorsement of specific symptoms, which appears to contribute to discrepancy of symptom cutoff scores with other illness parameters (Sprafkin et al. 2013). For example, in the case of ADHD combined type, agreement between impairment cutoff and symptom cutoff may well be increased if criteria for the latter were simply changed to a specific number of symptoms out of a total of 18 symptoms. Another example is the establishment of diagnostic criteria for spectrum disorders that would include all disorders with the same general class to include “not otherwise specified” (NOS) diagnoses. Here, impairment cutoff would pertain either to the entire spectrum or any disorder within the spectrum. Whether such strategies are even desirable is unexplored, and the case for redefining a nosological scheme is better made with a larger sample of children (e.g., youth with ASD plus co-occurring ADHD) and then to compare the validity of different parsing strategies with a broad range of external validators.

There are longstanding controversies in the ASD literature pertaining to the misidentification of co-occurring psychiatric symptoms. For example, there is evidence to suggest that pathogenic factors associated with ASD may alter their typical clinical presentation. It is also possible the defining symptoms of ASD may be misinterpreted as characteristics of other disorders; however, correlations of anxiety (Guttmann-Steinmetz et al. 2010), depression (Gadow, Guttmann-Steinmetz et al. 2012b) and schizophrenia (Gadow 2012) symptoms with the core features of ASD are generally low, particularly in youth with ASD without co-occurring ADHD, suggesting this may not be a serious problem when interpreting group data. Nevertheless, in the absence of consensus criteria for diagnosing co-occurring syndromes supported at least in part with empirically driven models of pathogenesis, it is difficult to know their implications for our results.

Research and Clinical Considerations

There are many seeming conceptual difficulties in determining how to measure impairment as well as its relation to psychiatric symptoms. Some of these difficulties include: the constructs are not independent; symptoms are defined in terms of their impact on or consequences for others (i.e., social deficits); and impairment is conceptualized and measured in different ways. Some instruments such as the Brief Impairment Scale (Bird et al. 2005) measure impairment directly by asking about interpersonal relationships, functioning at school, and use of leisure time. In psychiatric interviews such as the K-SADS-PL and the Diagnostic Interview Schedule for Children Version IV (DISC-IV; Shaffer et al. 2000), impairment is often assessed by a few questions after probing for symptoms within a diagnostic category. Finally, impairment can also be measured globally across multiple life domains. One of the most widely used global measures of impairment is the Children’s Global Assessment Scale (CGAS; Shaffer et al. 1984), which has been adapted for use with children with ASD (Wagner et al. 2007). These global measures were initially designed to yield one summary score regardless of the functional domains in which impairment occurred and the diagnostic categories in which symptoms were endorsed.

In research applications, the method chosen to measure impairment impacts study results. For example, Bird et al. (2000) investigated the relation between impairment questions at the end of each diagnostic category of the DISC-2.3, the CGAS, and a separate rating scale of impairment (the Columbia Impairment Scale, CIS). Prevalence rates in their epidemiological sample varied depending on how impairment was measured. The impairment questions imbedded in the DISC consistently identified more individuals than either the CGAS or the CIS.

Data from this and other studies of children with ASD (Leyfer et al. 2006; Witwer and Lecavalier 2010) suggest that a significant minority of children with ASD who are considered to be functionally impaired by their co-occurring psychiatric symptoms do not meet full criteria for a diagnosis. In real word clinical situations, it is difficult to believe that patients would be turned away because they lack all the prerequisite symptoms for a diagnosis. NOS diagnoses are likely used as a “safety net” for individuals who are impaired but without a sufficient quantity of symptoms (Angold et al. 1999). Up until now, most research and discussion about the nosological implications of psychiatric syndromes in children with ASD has focused on the difficulty distinguishing between their respective symptoms and altered presentations in children with ASD. The results of the present study suggest the long-neglected topic of impairment warrants consideration in future research as well.