The validity of joint attention assessment in school-aged children with ASD is unclear (Lord and Jones 2012). This study examined the validity of a parent-report measure of joint attention related behaviors in verbal children and adolescents with ASD. Fifty-two children with ASD and 34 controls were assessed with the Childhood Joint Attention Rating Scale(C-JARS). The C-JARS exhibited internally consistency, α= 0.88, and one factor explained 49% of the scale variance. Factor scores correctly identified between 88 and 94% of the children with ASD and 62–82% of controls. These scores were correlated with the ADOS-2, but not other parent-report symptom measures. The C-JARS appears to assess a unique dimension of the social-phenotype of children with ASD.

Joint attention is one of the more theoretically and clinically important dimensions of the social phenotype of Autism Spectrum Disorders (e.g. Charman 2004; Kasari et al. 2008; Lord and Jones 2012; Mundy 2016; Mundy et al. 1994). Five of the ten items of the Social Affect (SA) dimension of Modules 1 and 2 of the gold-standard Autism Diagnostic Observation Scale-2 (ADOS-2) assess joint attention behaviors. Moreover, evidence suggests these joint attention items constitute a distinct factor within the SA scale of these modules of the ADOS-2 (Gotham et al. 2007, 2008).

Joint attention refers to the perceptual and mental capacity to adopt a common frame of reference in order to share experience and process information about a common referent with other people (Mundy 2016). Joint attention can be measured in typical development by 4- to 6-months, and in the development of infants at risk for ASD by 8–9 months of age (e.g., Gredebeck et al. 2010; Ibanez et al. 2013; Mundy et al. 2007). Two types of joint attention are often measured. One is Responding to Joint Attention (RJA) or the ability of infants to follow the visual line of regard and/or pointing gestures of other people. The other is Initiating Joint Attention IJA or the ability of infants to direct the gaze and attention of other people (Initiating Joint Attention, IJA). Items measuring both types of joint attention are central to the early, preschool identification of children affected by ASD, or at risk for ASD (e.g., Gotham et al. 2007; Ibanez et al. 2013; Nygren et al. 2012; Ventola et al. 2007).

After the preschool period, though, the utility of the joint attention as a significant dimension of the social phenotype of ASD is controversial. While behavioral evidence of atypical RJA and IJA joint attention is striking in its presence in preschool development, it is not as clearly present in older children or adults with ASD (Lord and Jones 2012). However, this difficulty with observing the presence of this dimension later in life in ASD may reflect a limit in the conceptualization of joint attention, rather than its true absence.

Advances in theory and research suggest that the pointing and eye contact behavioral indicators of joint attention that are useful in the assessment of infants and toddlers may not continue to be as sensitive measures of individual differences in joint attention development as children mature. This is because the overt behavioral practice of joint attention in infancy is thought to lead to the development of internalized mental joint attention processes (Mundy 2016). These mental joint attention processes support a variety of social and learning functions in older children that are not necessarily indicated by overt pointing, showing, or eye contact. These social and learning functions include referential language use, cooperative behavior, social-cognition, and the capacity to focus on a common referent in order to learn from instruction, among others (e.g., Mundy 2016; Mundy and Sigman 2006).

Support for this assertion comes from a diverse array of research such as observations that indicate that children and adults use joint attention (e.g. gaze coordination) to disambiguate linguistic references, infer intentions, and enrich mutual understanding in social communication interactions (Lee et al. 1998; Shockley et al. 2009; Shulze et al. 2013; Tribushinina 2014). Other research indicates that joint attention plays a role in establishing a common perceptual and cognitive frame necessary to the development of cooperative and collaborative behavior in childhood (Wu et al. 2013), as well as children with ASD (Dykstra Steinbrenner and Watson 2015). Cognitive as well as social psychological research also suggest that the experience of joint attention enhances information processing within individuals and is integral to a sense of intersubjectivity and shared experience in adults and children (e.g., Bayliss et al. 2013; Boothby et al. 2014; Böckler et al. 2012; Kim and Mundy 2012).

Data from several longitudinal studies also now confirm the long hypothesized longitudinal continuity between infant joint attention and childhood theory of mind task performance (e.g. Brooks and Meltzoff 2015; Charman et al. 2000; Kuhn-Popp et al. 2015). Moreover, a recent imaging study indicates that the neurodevelopment of joint attention can be validly measured in childhood between and 8 and 18 years (Oberwelland et al. 2016). Data from imaging studies of adults and children also report significant overlap between the functional cortical networks that activate during performance on joint attention measures and the functional neural correlates that activate in association with social-cognitive task performance (e.g. Redcay et al. 2013).

A growing literature also reports measures and observations of the impact of joint attention on children and adults affected by ASD or the broad autism phenotype (BAP). Several of these studies suggest that the effect of joint attention on information processing is attenuated in older people affected by ASD as well as adults with the BAP (Edwardset al. 2015; Falck-Ytteret al. 2015; Mundy et al. 2016; Zhao et al. 2015). Imaging studies have also provided observation of atypical neurocognitive processing associated with joint attention task performance in school-aged children and adults with ASD (Caruana et al. 2014; Greene et al. 2011; Pelphrey et al. 2005; Redcay et al. 2012; Vaidya et al. 2011). Thus, research not only has begun to measure and describe the role of joint attention after infancy and the preschool period in typical development, but also suggests that atypical joint attention may be observable in people with ASD beyond their preschool years.

It remains unclear, however, whether a clinical index of symptoms of joint attention disturbance can be readily observed in people affected by ASD after the preschool period. All the literature cited above involves the use of basic research paradigms that do not lend themselves to the type of easily observed symptom presentation required of clinically valid and useful assessments. What is needed are ecologically and developmentally appropriate in-vivo measures of children’s and adolescents’ behaviors that that can be used to examine the hypotheses that joint attention remains a significant part of the social phenotype of ASD beyond the preschool period.

One approach to meet this need is to examine the degree to which parents can provide valid observations of the development of the joint attention in children and adolescents with ASD. Moreover, a strong test of the hypothesized developmental continuity of joint attention would be to determine if parents could rate joint attention development in verbal children with ASD who are not affected by comorbid intellectual disabilities. This is because school-aged children with ASD who exhibit less language development and are affected by intellectual disabilities may remain amenable to infant/preschool joint attention assessment by virtue of their delayed developmental status. On the other hand, approximately 68% of second-grade children with ASD are verbal and have IQs either in the borderline range or the typical range (Christensen et al. 2016). Infant/preschool measures of joint attention are less likely to be developmentally appropriate or sensitive to developmental differences in this subgroup of higher-functioning children with ASD (HFASD).

There are numerous valid symptom measures for school-aged children and adolescents with HFASD such as the revised Autism Diagnostic Observation Scale Modules 3 and 4 (ADOS-2, Lord et al. 2012), the Social Responsiveness Scale (SRS, Constantino et al. 2007), the Autism Symptom Screening Questionnaire (ASSQ, Ehlers et al. 1999), and the Autism Quotient (AQ, Baron-Cohen 2012). A survey of these instruments however reveals few if any items can be identified as specific to the assessment of joint attention behaviors. This not to say such items are not included. For example, Module 3 of the ADOS-2 includes items that assess reporting information about a non-routine event, providing information about self, asking about the examiner’s experience, using eye contact to regulate social interaction. These items may reflect spontaneous sharing of experience which is a fundamental feature of joint attention (Kasari et al. 1990; Mundy et al. 1992). However, these ADOS-2 items have, heretofore, not been explicitly recognized as potentially reflecting joint attention. This lack of such recognition likely contributes to the concern that joint attention is striking in its presence in preschool development, but is not clearly present in older children or adults with ASD (Lord and Jones 2012) and confusion about why a joint attention factor may be present in Modules 1 and 2 of the ADOS 2 but not appear to be present in Module 3 (see Gotham et al. 2006, 2007).

The uncertainty about the nature of joint attention in childhood motivated this study. Although, the conceptual foundation for the construct of joint attention is still being developed a major tenet of joint attention theory is that it has impact across the lifespan not just in the preschool period (Mundy 2016). In a paper published in this journal we reported results consistent with this hypothesis using a new information processing paradigm of joint attention development in 8 to 16-year-old children with ASD (Mundy et al. 2016). This present study was designed to test the life span hypothesis of joint attention in autism by examining whether a pool of valid parent report items could be identified to form another new and useful measure of the childhood development of joint attention for ASD research.

The Assessment of Joint Attention in School Aged Children

One comprehensive model of the construct of joint attention (Mundy 2016) suggests that there are at least three domains of social behavior that involve joint attention in childhood. Two of the domains, one verbal and one non-verbal, reflect spontaneously sharing interests with others. The notion that joint attention in ASD and typical development reflects spontaneous sharing of interests with others is supported by evidence that joint attention often involves the sharing of the emotional experiences of objects or events with other people (e.g. Kasari et al. 1990; Gangi et al. 2014; Mundy et al. 1992; Parlade et al. 2009). Historically, this conceptualization of joint attention was explicitly recognized in a prior version of the nosology of ASD. The DSM-IV and DSM-IV-R (APA, 1994,2000) included “a lack of spontaneous seeking to share enjoyment, interests, or achievements with other people, indicated by a lack of showing or pointing out objects of interest” as one of the four core social behavioral diagnostic items of ASD. This item operationalized spontaneous sharing of experience in terms of observations of initiating joint attention behaviors, such as showing. The presence of this joint attention symptom item and its conceptualization, however, is not clearly recapitulated in the current version of the nosology (DSM-V, APA, 2013). Nevertheless, sharing experience with others has also long been recognized as a fundamental function of joint attention development in typical children (e.g., Bates et al. 1979; Rehingold et al. 1976; Venezia et al. 2004), as well as atypical joint attention development in ASD (e.g., Charman 2004; Kasari et al. 1990; Mundy and Sigman 2006).

Research on joint attention indicates that children and adults continue to frequently use nonverbal gaze following and gaze directing to establish a common point of view in order to spontaneously share experience and information with other people (Mundy 2016). So, it is plausible that parents may be able to provide valid observations about children’s tendency to use these nonverbal joint attention behaviors to share experience with others. Parent report of these types of behaviors would have clear face validity for the assessment of joint attention in children.

With respect to measures of the tendency to verbally spontaneously share experience with others we have hypothesized that the social-psychological phenomena of capitalization is part of the expression of joint attention in older individuals (Mundy 2016; Mundy and Newell 2007). Capitalization occurs when people (e.g. marital partners) initiate joint attention with a social partner in reference to a positive experience they’ve had. For example, a wife may come home from work and recount praise received in the course of her day to her husband. Gable et al. (2004) and Boothby et al. (2014) have provided evidence that, beyond the positive impact of the event itself, the spontaneous sharing of a positive daily experience followed by an attentive response by the social partner has a unique and positive impact on the affect, and sense of relatedness, of the social dyad.

A third domain follows from the evidence that joint attention plays a vital role in establishing a common perceptual and cognitive frame necessary to the development of joint action, as practiced in cooperative and collaborative behavior in typical children and children with ASD (e.g. Mundy 2016; Wu et al. 2013). It is plausible that parents can provide valid reports of individual differences in cooperative and collaborative behavior tendencies that would, in part, contribute to a measure of the latent construct of joint attention in childhood.

Based on these hypotheses, and our expertise in the study of joint attention (e.g. Mundy 2016), a 60-question item pool measuring three theory-based domains of joint attention behavior in childhood were generated by the first and second author. Ten percent of these items were gleaned from existing instruments, such as the ADOS-2 Modules 3 and 4 (Lord et al. 2012); Autism Symptom Screening Questionnaire ASSQ (Ehlers et al. 1999); Autism Quotient (AQ, Baron-Cohen 2012); and the Social Responsiveness Scale (SRS, Constantino et al. 2007). However, the majority were developed de-novo based on the research literature on joint attention and related construct briefly describe above, and reviewed in detail by Mundy (2016). These were combined to create a preliminary version of the Childhood Joint Attention Rating Scale (C-JARS). A study was then designed to test three hypotheses about the psychometric characteristics of this scale.

The first hypothesis was that the items of the C-JARS would exhibit a factor structure and sufficient evidence of internal consistency of at least one factor based scaled score to support the hypothesis that the C-JARS provides a reliable measure a dimension of joint attention related social development in children and adolescents. The second hypothesis was that if the dimension of the C-JARS was sensitive to joint attention development it should be sensitive to differences in the social development of 8- to 16-year-old children with HFASD in comparison to children with typical development as well clinical comparison children with clinical elevations of ADHD symptoms. The latter were included because of reports of comorbidity in the presentation of ADHD symptoms among children with HFASD (Gargaro et al. 2011) and associated problems in differential identification in school-age children with ADHD and HFASD (e.g. Ehlers et al. 1999). The third hypothesis was that the C-JARS would not be redundant (e.g. strongly correlated) with other current parent report measures. That is the C-JARS it would reflect a unique dimension of disturbance in children with HFASD that is not well measured by other contemporary parent report measures of childhood social symptoms of ASD. This hypothesis was based on the observation that few of the behaviors or symptom described in items of the C-JARS appear in other contemporary parent report ratings for children with ASD.

Methods

This research was conducted in compliance with the appropriate university Institutional Review Board, and written consent and assent was obtained from parents and participants before gathering any data.

Eighty-six children between the ages of 8 and 16 who were part of a larger study longitudinal study of academic and social development in school-aged children with HFASD participated in this study (see McIntyre et al. 2017 for details). The HFASD sample included fifty-two children with ASD (42 boys and 10 girls) recruited from local schools with diagnostic confirmation using the Autism Diagnostic Observation Schedule, Second Edition (ADOS-2; Lord et al. 2012) administered by a post-doctoral fellow with ADOS-2 research reliability training. The majority of the HFASD sample had an IEP or 504 Plan and spent much or all of their school day in a general education classroom. All the children in the HFASD sample had IQs greater than 75 on the Wechsler Abbreviated Intelligence Scale (WASI-2, Wechsler 2012). Parent report data of ASD symptom presentation was also gathered with the Social Communication Questionnaire-Lifetime version (SCQ, Berument et al. 1999), the Autism Spectrum Symptom Questionnaire (ASSQ, Ehlers et al. 1999), and the Social Responsiveness Scale (Constantino et al. 2003). ADHD symptom data were gathered with parent report on the Conner’s-3 parent report (Conners 2010). The Control/Comparison (CC) sample was comprised of 34 age and gender matched children without HFASD who were also participating in the longitudinal study (25 boys, 9 girls). None of these children received parent report scores that revealed clinical levels of ASD symptoms on the SCQ, ASSQ or SRS.

The longitudinal study included comparison samples of children with typical development (TD) and children with community diagnoses of ADHD. The control sample in this study included 23 children with TD and 11 from the ADHD sample. The latter displayed elevated parent reports of total ADHD scores on the Conners-3 (T- scores 60 to 79). Even with the inclusion of this subgroup in the comparison sample, parents report indicated that the HFASD group presented with a higher rate and intensity of ADHD symptoms than did the control/comparison sample (see Table 1). Twenty-six of the children in the HFASD sample were prescribed medication for ADHD symptoms (N = 18) or other medications (N = 8, for mood, anxiety and/or self-regulation) and 12 of the children in the control sample were prescribed stimulant medication for ADHD.

Table 1 Demographic data for the diagnostic groups
Full size table

The preliminary data analyses indicated that the diagnostic groups differed significantly on IQ, as well as all parent report measures of ASD symptoms (see Table 1). Exclusionary criteria for all participants included an identified syndrome other than ASD or ADHD (e.g., Fragile X), significant sensory or motor impairment (e.g., visual impairments), a neurological disorder (e.g., epilepsy, cerebral palsy), psychotic symptoms (e.g., hallucinations or delusions), or any major medical disorder reported by parents that significantly affected the child’s behavior, or involved four or more week’s absence from school in the most recent academic year.

Diagnostic & Cognitive Measures

The Autism Diagnostic Observation Schedule, Second Edition (ADOS-2; Lord et al. 2012) is a semi-structured “gold standard” diagnostic assessment for ASD shown to have strong predictive validity against best estimate clinical diagnoses (Charman and Gotham 2013). Trained personnel administered Module 3 or 4 to confirm ASD diagnosis through evaluation of two core domains: Social Affect (SA) and Restricted and Repetitive Behavior (RRB). The Module 3 algorithm yielded a raw sub-score for SA and for RRB that combined to create the Total Score. The Social Communication Questionnaire-Lifetime version (SCQ; Berument et al. 1999) provided a 40-item parent report rating developmental social communication, and stereotyped and repetitive behavior symptoms of ASD in children 4 years and older. The High-Functioning Autism Spectrum Screening Questionnaire (ASSQ, Ehlers et al. 1999) is a 27-item checklist screener and one of the few measures with demonstrated test–retest reliability (Parents, 0.96, Teachers 0.94) and diagnostic validity for discriminating children with HFASD from other groups. The Social Responsiveness Scale (SRS, Constantino et al. 2003) is a 65-item parent-report index of social behaviors in children with autism or typical development.

The Conners-3 (Conners 2008) is a parent report checklist of children’s current behavioral symptoms of ADHD. The Conners-3 DSM-IV-TR Symptom Scales for Inattentive Type and Hyperactive-Impulsive Type represent the main clinical constructs of the DSM by asking parents to rate their child on items that are close approximations of each of the DSM-IV-TR symptoms for these subtypes. Age- and gender-normed T-scores (M = 50, SD = 10) allow comparison of an individual’s level of symptoms with that of same age and gender peers.

The WASI-2 (Wechsler 2012) provides an estimate of verbal and nonverbal cognitive ability. Two verbal subtests, Vocabulary and Similarities, measured expressive vocabulary and abstract semantic reasoning and formed the verbal composite (VIQ). Two non-verbal subtests, Block Design and Matrix Reasoning, measured spatial perception, visual abstract processing & problem solving with motor and non-motor involvement and formed the performance composite (PIQ). Combined, the four subtests yielded an age-normed standard score (M = 100, SD = 15) measurement of full-scale IQ (FIQ).

The Childhood Joint Attention Rating Scales (C-JARS)

The 60 parent report items of the C-JARS were developed to reflect three hypothetical types of behavior expression of the construct of joint attention in childhood. One verbal and one non-verbal set of items assessed spontaneously sharing experience with others, which was one of the four social behavioral diagnostic items for ASD in the previous nosology of Diagnostic and Statistical Manual-IV-TR, APA, 2000. The nonverbal cluster of spontaneous sharing items was selected based upon the primary body of research on joint attention, which indicates that children and adults frequently use nonverbal gaze following and gaze directing to establish a common point of view in order to spontaneously share experience and information with other people. An example of an item from this Nonverbal Sharing Interests Domain is provided in Table 2.

Table 2 Examples of C-JARS items, social functions, & scoringa
Full size table

As noted in the introduction, the verbal cluster of items in this domain were developed based on the social-psychological literature on capitalization (e.g., Gable et al. 2004). These items assessed the tendency of children to spontaneously share positive events or interests with their parents or peers. Table 2 provides an example of a prototypical capitalization, or verbal spontaneous sharing, item of the C-JARS.

A third cluster of items on the C-JARS was developed based on the previously noted evidence of the association between joint attention and the development and maintenance of cooperative and collaborative behavior in typical children and children with ASD (e.g. Wu et al. 2013). This item domain is referred to as Joint Action on the C-JARS and an exemplary item of this domain also appears in Table 2.

Procedures

Parents were asked to complete the C-JARS, SCQ, AQ, SRS, ASSQ and Conner’s during one of two visits to a university child assessment laboratory as part of their child’s participation in a longitudinal study of academic and social development. The C-JARS was developed at the beginning of this study but was not completed in time to administer at the onset of the longitudinal assessment of the full sample of 164 children (See McIntyre et al. 2017). Therefore, the C-JARs was administered to a subsample of 86 families (54%). There were no differences in the age, gender ratio, IQs or symptoms presentation in the C-JARS sample versus participants who did not receive that C-JARS. Nevertheless, the Diagnostic Group differences on IQ in C-JARs sub-sample reached significance. The participants were between 8- and 16- years old at the time of the first visit when data on the ADOS-2, WASI, SCQ, SRS, and ASSQ were collected. Parents completed the C-JARS for their children during the second visit, which occurred fifteen months after Session 1 (+/1 thirty days). The C-JARS was developed in that 15-month interval.

Results

Data Analyses Plan

Double data entry was used to check the data base accuracy of all variables. Data on skew, kurtosis and Q-Q plots were examined for significant distribution anomalies across measures and groups (Kline 1998; Cohen 1988) including the scores derived for the C-JARS. However, no significant violations of assumptions of normal distributions were detected for any measures. Confirmatory factor analysis (e.g. Floyd and Widaman 1995; Thompson and Daniel 1996) with Varimax and Promax rotations were used to examine the simple factors structure of the C-JARS. Both orthogonal rotation (Varimax) and oblique rotation (Promax) were used to best estimate if the simple factor structure of the C-JARS was best represented by a single factor or multiple factors (Thompson 2004). After identifying the factor structure of the measure, scaled scores for factor items loadings greater than 0.40 were generated using the Bartlett method. Only items with loadings of 0.40 or greater were considered to make a significant contribution to a factor (Velicer and Fava 1998). The factor analyses were computed for the entire sample to generate a C-JARS comparable standard score (z-score) across all participants. Factor analyses were also conducted within ASD and control sample to examine if the C-JARS exhibited a similar factor structure across Diagnostic Groups. Reliability (internal consistency) of the factor based scale scores from the C-JARS were examined with Chonbach’s alpha for the total sample and within each diagnostic group.

ANCOVA with IQ as a covariate was used to test for expected Diagnostic Group differences in parent report on the C-JARS. Logistic regression was employed as a method to provide more detailed information about construct validity (Coste et al. 1995; Peng et al. 2002). In this case it was used to estimate the number of children for whom the C-JARS provided an index of a characteristic of social developmental that is specific to children with HFASD in the study sample.

Finally, parametric correlations were conducted to test the concurrent convergent and divergent validity of the C-JARS with respect to: (a) providing social symptom information that was not provided by other parent report measures of symptoms in children with HFASD but, (b) did display concurrent validity in the guise of a positive relation with observations of symptoms on the ADOS-2.

Factor Analyses

A factor analysis with Varimax rotation of all 86 sets of parent responses on the C-JARS resulted in the identification of one primary factor, Eigenvalue = 26.82, which accounted for 48.7% of the item covariance. The majority of C-JARS items load on this single factor except for four items that had lower factor loadings than all other items (<0.40). These were eliminated leaving 56 items on the C-JARS factor. Three other possible secondary factors were also revealed in this confirmatory factor analysis with eigenvalues ranging from 2.0 to 3.1. However, each factors was characterized by only one or two items with a factor loadings of 0.40 or greater (range = 0.40–0.45). The nature of these factors could not be interpreted in terms of identifiable joint attention factors and were not considered further. The Promax rotation solution was nearly identical. It identified one primary factor that accounted for 49.6% of the variance in C-Jars items, but also eight secondary factors that accounted for 1–3.2% of the variance across C-JARS items. Again, these factors were difficult to interpret because each one was characterized by only a few items (3 or less) with factor loadings of 4.0 or greater (range = 0.41–0.46).

Exploratory analyses to examine the similarity or differences in the factor structure of the 56 items C-JARS in the HFASD and the control sample in this study. The factor analysis of the C-JARS in the ASD sample yielded a primary factor, Eigenvalue = 22.3, which accounted for 40% of the item covariance. All 56 items of the C-Jars loaded on this factor (range = 0.41–0.78) with an average item loading of 0.62. The Promax rotation solution yielded a similar picture of the simple structure of the C-JARS with one primary factor that involved all items and accounting for 39% of the variance. The factor analysis of the C-JARS in the control sample also yielded a primary factor, Eigenvalue = 22.3, which accounted for 48% of the item covariance. All 56 items of the C-Jars loaded on this factor in the control samples as well (range = 0.45–0.88) with an average item loading of 0.69. Again, the Promax solution was nearly identical to the Varimax solution in that it identified one primary factor that accounted for 52% of the variance.

A comparative Varimax factor based Joint Attention Z-score was computed for all participants from the analysis of the 56 item C-JARS using the Bartlett method. The internal consistency of this single factor-based scale score for the C-JARS was Cronbach’s α = 0.86, p < .001 for the total sample. Separate comparable consistency estimates were obtained for analyses only within the HFASD sample, α = 0.84, p < .001, and only within the Control sample, α = 0.89, p < .001. All the 56 items for the C-JAR are available in the supplemental table on line or from the first author.

Diagnostic Group Differences

A 2 (Diagnostic Groups) X 2 (Age Groups, < 11.5 year >) X 2 (Gender) Analysis of Covariance (ANCOVA), controlling for Diagnostic group differences in full scale IQ, was the conducted to examine the hypothesis that parents’ reports indexed by the C-JARS factor based joint attention scaled score would differ across children in the HFASD or Control sample. The results revealed a significant effect for Diagnostic Group, F (1, 77) = 21.8, p < .001, eta2 = 0.21, such that the control sample exhibited significantly more positive factor-based Z-scores on the C-JARS than did the HFASD sample (see Table 3). The eta 2 effect size estimate of 0.21 is equivalent to a Cohen’s d effect size estimate of 1.03 (Cohen 1988)Footnote 1.

Table 3 Means and standard errors for diagnostic group, diagnostic group X age and diagnostic group X gender comparisons of the C-JARS factor based Z-scores
Full size table

No main or interaction effects involving Age Group were observed in these analyses. Alternatively, evidence of a significant effect for Gender, F (1, 77) = 8.50, eta2 = 0.10, and a significant Diagnostic Group by Gender interaction effect were observed, F (1, 77) = 4.02, p < .05, eta2 = 0.10 (Table 3). Follow-up analyses revealed that a significant effect for Diagnostic Group was observe when only the data for boys were analyzed, F (1, 64) = 8.24, p < .006, eta2 = 0.14, such that boys in the Control sample displayed more positive C-JARS scores than boys in the HFASD group. In addition, an appreciably stronger Diagnostic Group effect estimate was observed for the advantage of the girls in the Control sample on the C-JARS relative to girls in the HFASD group, F (1,16) = 12.36, p < .003, eta2 = 0.44 (Table 3).

Correlational analyses were conducted as an approach to understanding the possible impacts of Age, IQ, gender, ADHD symptoms or medications (1 = none, 2 = stimulant and 3 = other) on the C-JARS scores within the Diagnostic Groups. The results of the analyses revealed no significant associations in the ASD sample, C-JARS with: Age, r = − .04, Gender, r = .13, VIQ, r = .12, PIQ, r = .17, Conner’s Inattention Scale, r = .06, Conner’s Hyperactivity Scale, r = − .13, or Medications, r = − .10. There was one significant correlation observed in the control sample, C-JARS with: Gender (point-biserial coefficient), r = .54, p < .001. All other correlations ranged from 0.22 to − 0.23, ps > 0.19. The observation of a correlation with gender in the Control, but not the HFASD sample, converged with those of the ANCOVAs and provide useful detail. With respect to the gender correlation in the Control sample all nine girls obtained C-JAR factor Z-scores above 0. The Z-scores of scores of the boys were more evenly distributed with thirteen boys scoring scores above 0 and ten boys scoring below 0. Conversely, only 2 of the girls in the HFASD sample obtained C-JARS scores above 0.

As a follow-up to the ANCOVA a logistic-regression analyses was used to provide more detail on the Diagnostic Group difference in terms of the sensitivity and specificity of the C-JARS to differences Joint Attention factor scores in the ASD sample. These analyses tested three models. In the first model, the C-JARS was the lone discriminant variable. The second model tested whether IQ played a role in C-JARS sensitivity and specificity for children with HFASD. The third model examined the possible influece of comorbidity of attention problems and included the combination of the C-JARS and total Conner’s ADHD symptoms.

Analyses of the first model revealed that the sensitivity of the C-JARS Joint Attention score (correct identification of the HFASD sample) was 46 out of 52 (88%) and specificity, or correct identification of members of the Control sample, was 21 out of 34 (62%), χ2 = 19.35, p < .001, p < .001. Analyses of the second logistic regression model that included IQ with the C-JARS did not lead to a change in the signal detection characteristics. A third model that combined parent report on the C-JARS with parent report of the combine Conner’s 3 Inattention and Hyperactivity score, however, led to slightly improved sensitivity 94% and substantially improved specificity 82% of Diagnostic Group identification, χ2 = 65.11, p < .001.

The signal detection characteristics of the C-JARS were also examined separately by gender because of presence of a Diagnostic Groups by Gender interaction on C-JAR scores.

C-JARS scores alone were associated with 90% sensitivity but only 48% specificity for HFASD and Control group boys, χ2 = 9.07, p < .003. With the addition of ADHD scores these signal detection characteristics changed to 93% sensitivity and 76% specificity, χ2 = 45.70 p < .001. The initial model for girls yielded a sensitivity estimate of 80% and specificity estimate of 78%, χ2 = 13.76, p < .001. The addition of the ADHD symptom scores to the model changed the sensitivity and specificity to 100%, χ2 = 26.29, p < .001.

Convergent and Divergent Construct Validation

Finally, correlation analyses were conducted to examine the possible construct overlap of parent report on the C-JARS, with parent report on the Lifetime SCQ, the ASSQ, and SRS in the HFASD sample. In addition, the association of parent report C-JARS with the objective observations of symptoms on the ADOS-2 in the HFASD sample was also examined. These analyses revealed that the scores from the SCQ, SRS, and ASSQ were all significantly inter-correlated within the HFASD sample, SCQ to ASSQ, r =. 53; SCQ to SRS to r = .58; SRS & ASSQ, r = .74, all ps < 0.001. These parent report scales were also significantly correlated within the Control sample.

Alternatively, analyses revealed that none of the correlations between the parent report measures of the C-JARS and total scores of the other parent report measures of ASD symptoms were significant in either diagnostic group (see Table 4). In terms of subscales of the SCQ the only significant correlation with C-JARS scores was observed for the SCQ-Social items subscale score in the control sample, r = − .37, p < .03, the comparable coefficient in the HFASD sample was, r = − .12. Thus, as expected there was minimal evidence of construct overlap in the pattern of associations between the C-JARS and other parent report symptom measures because the items of the C-JARS assess behaviors and symptoms that are not described by the items included in the other parent report measures included in this study.

Table 4 Group comparison of the correlations of the C-JARS joint attention factor based scalescores with other parent report symptom measures, and ADOS-2 Scores
Full size table

The construct validity of the C-JARS, however, was supported by evidence that higher, more positive C-JAR scores in the HFASD were significantly associated with lower objective symptom observations on the ADOS-2 Comparison and Total symptoms scores (see Table 4). This association appeared to be carried more so by the C-JARS correlation with Social Affect (SA) Factor score, r = −.24, p < .04, one-tailed, rather than the RRB factor, r = − .13, p < .20, one-tailed. The strongest associations between the C-JARS and the items of the ADOS-2 SA factor were observed for, “Directed Facial Expressions, r = − .28, p < .025, one-tailed; “Shared Enjoyment”, r = −. 23, p < .06, one-tailed; Insight into Typical Social Relations, r = − .21, p < .08, one-tailed, and “Quality of Overall Rapport”, p < − .22, p < .06, one-tailed.

Discussion

Problems with joint attention have long been considered to be a pertinent, if not primary, dimension of the social phenotype of ASD (e.g. Charman 2004; Kasari et al. 2008, 1990; Mundy and Sigman 1989). Joint attention items play a primary role in diagnostic instruments for younger children (Gotham et al. 2007, 2008) as well as early screening instruments (Nygren et al. 2012; Ventola et al. 2007). More recently it has become clearer that amelioration of joint attention disturbance is an important target of effective early intervention (Kasari et al. 2008; Gulsrud et al. 2015). However, there has been little formal recognition of the role of joint attention in the current DSM 5 or ICD nosologies of ASD.

The reasons for this paradox are not completely clear. However, Lord and Jones (2012) alluded to lack of evidence of joint attention’s impact beyond the preschool years of life as a major limit on the utility of this construct in diagnosis and research on autism. However, recent research indicates abnormal neurodevelopment specific to joint attention continues to be observed into adulthood (Pelphrey et al. 2005; Redcay et al. 2013). Experimental behavioral paradigms also suggest that joint attention can be assessed in school-aged children with ASD and that atypical joint attention continues to have an impact on the social-communication, learning and information processing of these children (e.g. Bockler et al. 2014; Dykstra et al. 2015; Mundy et al. 2016; Swanson et al. 2013). This study attempted to contribute to this literature by examining whether it was possible to develop a parent report measure that was sensitive to differences in joint attention related behaviors among school aged children with HFASD.

The results of the study were consistent with the hypothesis that parent report on a measure of childhood joint attention behaviors, the C-JARS provides an index of a valid social symptom dimension of 8- to 16- year-old higher function children with ASD. Parents reported that children with HFASD displayed fewer C-JARS joint attention behaviors than children without HFASD. The difference between the samples was such that the C-JARS parent report displayed good sensitivity and moderate specificity for diagnostic identification of children in HFASD and control/comparison samples. This was the case for both boys and girls with HFASD in this study. Convergent evidence of validity was also provided by the observation that parent report on the C-JARS was correlated with independent objective observations of children’s social behaviors on the ADOS-2, especially with the Social Affect factor. Few items were gleaned from the ADOS-2 in the development of C-JARS items. Nevertheless, the descriptors of some of the algorithm items of Modules 3 of the ADOS-2 appear to be more specifically related to the dimensions of the C-JARS than others. These include “sharing enjoyment”, “reporting of events”, and possibly “unusual eye contact”. However, to our knowledge none of these items, nor any other Module 3 SA factor items, have been discussed in terms of their possible specific assessment of the construct of joint attention (e.g. Gotham et al. 2007, 2008; Lord et al. 2000). The data analyses indicated that none these items were singularly responsible for the significant association of the ADOS-2 with the C-JARS. Thus, the data can only be interpreted to suggest that the latent SA construct of the ADOS-2, rather than only a few specific items, was related to the C-JARS operationalization of joint attention in children and adolescents with ASD in this study.

In addition evidence of divergent construct validity of the C-JARS was offered by observations that indicated there was minimal evidence that it was associated with the symptom construct (s) measured by parent report on the SRS, ASSQ or SCQ. This observation reflects a set of null findings and, therefore, must be interpreted with caution. Nevertheless, this pattern of null findings is consistent with the hypothesis that contemporary parent report measures of social symptoms in school aged children with ASD do not contain many if any measures that reflect joint attention development. The absence of joint attention items may have contributed to the notion that evidence for the impact of joint attention in childhood has been lacking because we had not developed measurement instruments that were designed to be sensitive to the childhood expression of this construct (Mundy 2016). At the very least these data suggest that a valid social symptom dimension of higher-functioning children with ASD, which is assessed by the C-JARS, is absent from contemporary parent report symptom screening instruments in use with school-aged children.

Other evidence of the measureable effects of the impact of joint attention in children and adolescents with ASD has also been presented in a recent experimental study. Using a novel VR paradigm (Kim and Mundy 2012), Mundy et al. (2016) reported evidence of significant differences in the impact of joint attention on information processing in children with HFASD than in control children. Together with the more clinical behavioral observations afforded by the C-JARS, these two new measures have provided data that are consistent with the hypothesis that joint attention may be a significant social dimension of ASD that exhibits developmental continuity and influence beyond the preschool period in ASD.

The term “social dimension” is used advisedly for joint attention because the construct exhibits the characteristics of a ‘dimension’ of developmental psychopathology as described in the current Research Domain Criteria (RDoC, Cuthbert and Insel 2013). The RDoC initiative identified dimensions of psychopathology on the basis of: (a) strong evidence of measurement reliability and validity, across typical and atypical groups, and (b) strong evidence that a construct/dimension maps on to a specific biological system such as a brain circuit (Cuthbert and Insel 2013). There is ample evidence of the reliable and valid assessment of joint attention in studies of infants with typical development (e.g. Mundy et al. 2007), and in infant and preschool diagnostic and intervention studies of ASD (e.g. Gotham et al. 2007; Ibanez et al. 2013; Kasari et al. 2008). The results of this study, and that of Mundy et al. (2016), add to this evidence for children and adolescents. Strong evidence that a construct/dimension of joint attention maps onto a specific brain circuits is beyond the scope of this paper but has been presented elsewhere (Mundy 2016; Mundy and Jarrold 2010).

Limitations

Of course, not all the results of this study provided unequivocal evidence of the validity of joint attention measurement in school-aged children with HFASD. The estimates of the specificity of the C-JARS in this study were modest. This suggests that the social dimension assessed by the C-JARS may be perturbed in children who are not affected by ASD. Specifically, the data in this study raised the possibility the lower scores on the C-JARS may be associated with elevated attention and hyperactivity problems in some children. Evidence of this was provided by the observation that combining information from the C-JARS and the Conner’s 3 parent report data on ADHD improved the specificity of the discrimination of the diagnostic groups (62–82%) without decreasing the level of sensitivity achieved by the C-JARS alone.

Given the sample size of this study it should also be understood that the data in this study do not speak to the clinical utility of the C-JARS. The sensitivity and specificity of this measure would need to be examined in multiple studies with larger samples before any conclusion with regard to its clinical utility could be advanced. However, the goal of this study was not clinical in nature. Rather it was to test the hypothesis that that construct of joint attention could be measured via parent report in school-aged children with ASD. The results provided enough support for this possibility to suggest that further research on measures like the C-JARS is warranted and potentially useful in the science of ASD. Because of the sample-size it is also important to recognize that this study cannot provide definitive data on the factor structure of the C-JARS. That will need to be determined in studies with much larger samples.

Perhaps the most important limit of this study is that it does not provide conclusive evidence that joint attention, per se, is a dimension of social development in school-aged children and adolescents with ASD. Although the development of the C-JARS was guided by theory and research on a life span approach to the conceptualization of joint attention (e.g. Böckler et al. 2012; Edwards et al. 2015; Mundy 2016; Oberwelland et al. 2016), there is no guarantee that the developers were sufficiently accurate (or clever) in choosing items that truly reflect joint attention development. Indeed, researchers may well challenge the assertion that the behaviors rated on the C-JARS validly reflect the construct of joint attention. However, with the availability of the C-JARS this is now an empirical question that can more directly be approached in future research. Research directed to address this issue may well deepen the current understanding of the nature of joint attention in autism syndrome development.

The extant evidence that each of the categories of behaviors assessed on the C-JARS are associated with joint attention development (e.g. Mundy 2016) notwithstanding, there are several approaches to the succeeding empirical examination of the validity of the hypothesis that the C-JARS measures joint attention. A non-exhaustive list here would include a test of the hypothesis that the C-JARs is related to measures of childhood social cognition in children with HFASD, as would expected by joint attention theory (Mundy 2016; Redcay et al. 2013). Another approach would be test the hypothesis that variation in preschool joint attention should predict individual differences in school age C-JARS scores. A third approach would be to examine the relations between the C-JARS and the classroom engagement of students with ASD. Theory explicitly suggests that joint attention is necessary for adaptive engagement with learning in any instructional context (Mundy 2016), and some classroom engagement measures for ASD rely on the construct of joint attention (e.g. Dykstra et al. 2015).

Other limitations of the study included the lack of a standardized measure of adaptive function in the assessment of “high functioning” status. In this study we relied on the report that the participant was able to spend much or all of their day in a regular education classroom as a gross index of adaptability. Another limitation is that language level was only assessed and controlled in this study with the two verbal scales of the WASI. A more detailed analysis of how language differences among higher functioning children may effect parent ratings on this C-JARS is warranted. Third, the ADOS-2 was not used to rule out symptom presentation of ASD in the control samples, instead parent report on the ASSQ, SRS, and SCQ were used for this purpose. The more rigorous use of the ADOS-2 to rule out ASD in the controls may be advisable in future studies. Finally, other than ADHD symptoms, the study did not control for other high frequency psychiatric comorbidities of ASD, such as anxiety or affective disorders and the study employed unequal sample size design. Both of these aspects of the methods should be recognized as possible limitations on the interpretation and/or power of this study.

In summary, these limitations suggest that additional research on C-JARS is necessary to consider its validity. The C-JARS should not at this time be considered to be a clinically valid instrument because the sample is too small for a definitive psychometric appraisal. The sample of this study also did not allow for a comprehensive appraisal of age effects on item response. Moreover, even though the results raised the possibility that the C-JARS maybe especially sensitive to symptoms in girls with HFASD symptoms, the relevant sample size limits any conclusion to be drawn from this pattern of results. So, this report provides a start, but only that. Consequently, the C-JARS should be considered to be an experimental measure rather than one that has current clinical utility. As an experimental measure it is freely available from the first author for use in research.

These caveats notwithstanding, the data in this study are consistent with the possibility that a measure of behaviors that reflect a “lack of spontaneous sharing of experience” may provide an index of a valid and unique dimension of the social phenotype of ASD in school-aged children. The assessment of this dimension figured prominently in the description of the social phenotype of ASD in DSM-IV. Indeed, the DSM-IV item that assessed this dimension of the phenotype was, in large part, reliant upon preschool joint attention measures such as showing gestures that served the nonverbal function of sharing interests. The results of this study suggest that it may be useful to reconsider the inclusion of this dimension in future ASD nosological revisions, especially if it was relinquished due to the perception that it was not measureable beyond the preschool period of development.