Challenges with social communication are a core symptom of autism spectrum disorder (APA, 2013), which is estimated to affect 1 in 54 children in the United States (Maenner et al., 2020). The American Speech Language Hearing Association defines social communication as the “rules for how we use language in different situations and with different people” (ASHA, 2020). This includes using communication for different pragmatic functions (e.g., greeting, commenting, questioning), adjusting communication for varying audiences and settings, and following social rules for communicating (e.g., giving people personal space, taking turns speaking, making eye contact). Even when structural language skills (vocabulary, sentence structure, receptive language) are intact in children with autism, use of communication skills within social settings is often impaired relative to societal expectations (Carpenter & Tomasello, 2000; Jones & Schwartz, 2009; Rubin & Lennon, 2004). The social-pragmatic aspects of communication characteristic of autism include stereotyped/scripted language, restricted/repetitive conversation topics, difficulties with perspective taking, maintaining topics, or adjusting communication to social contexts, understanding and use of nonverbal communication, and challenges understanding non-literal uses of language (APA, 2013; Rubin & Lennon, 2004).

Although several assessment tools exist for identifying speech and structural language impairments in children, few of them measure the social-pragmatic communication challenges common in autism. Measures that do assess social-pragmatic communication are often time-intensive (e.g., Autism Diagnostic Observation Schedule-2, Lord et al., 2012) or are more diagnostically focused for autism and therefore, lack information about social communication that is informative for goal writing and intervention progress monitoring (e.g., Social Responsiveness Scale, Constantino & Gruber, 2005). Measuring social-pragmatic communication skills is further complicated by the need to assess these skills within a variety of everyday social contexts which are not available within a clinical assessment environment (Bishop & Baird, 2001). Parent or teacher report of the child’s social-pragmatic behavior across contexts is one way to overcome this barrier. This study examines the psychometric properties of an existing caregiver-report measure of social-pragmatic communication skills, providing an updated report of the utility of this measure for clinicians and researchers.

The Children’s Communication Checklist-2, United States Edition (CCC-2; Bishop, 2006) is a 70-item caregiver rating scale of 4 to 16-year-old children’s communication skills. It is designed for children who speak in complete sentences and whose primary language is English. Caregivers rate each item by the frequency with which the child demonstrates the communicative behavior described. The CCC-2 was designed to identify children with characteristics of (a) speech language impairment, (b) autism, and/or (c) pragmatic language impairment. In a review of assessment methods for pragmatic language, Adams (2002) concluded that the CCC (Bishop, 1998) was the only pragmatic language checklist with satisfactory psychometric properties (internal consistency and interrater reliability); and since that time the CCC, and subsequent second edition, have been widely used.

The 70 CCC-2 items are added into 10 subscales, combinations of which are summed to yield three domains: the language domain, which consists of communication features commonly seen in children with Specific Language Impairment (SLI; see Bishop, 2017 and Volkers, 2018 for current views on SLI and Developmental Language Disorder), the pragmatic domain, which includes areas that impact language use in various social contexts rather than structural language, and the autism spectrum disorder domain. The General Communication Composite (GCC) may also be calculated from the subscale scores, which is indicative of the overall level of disordered communication a child demonstrates compared to norms. Finally, the Social Interaction Difference Index (SIDI) is indicative of the level of disordered structural vs. social-pragmatic communication skills a child demonstrates.

There are multiple reasons that an updated psychometric analysis for the CCC-2 is warranted. Although the CCC-2 manual reported strong internal consistency (0.69–0.85 across subscales; 0.94–0.96 for the GCC across age groups) and test–retest reliability (0.86–0.96 across age groups) data when it was published in 2006, it did not examine convergent or discriminant validity. Also, items on the CCC-2 were organized conceptually into the three domains or 10 theoretical subscales, but their organization was not confirmed through factor analysis. Furthermore, the clinical autism sample was modestly sized (N = 62) and internal consistency of the measure was not reported specifically for this population. Societal awareness of autism as well as publication of the 5th edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-5; APA, 2013) which grouped Asperger’s and pervasive developmental disorder – not otherwise specified (PDD-NOS) diagnoses into “autism spectrum disorder” suggest that updated psychometric information on the children on the autism spectrum is needed. Finally, no known study has examined the psychometrics of the second edition of the CCC (CCC-2) as reported by teachers. This manuscript will report an update and expansion of the psychometric analyses in the CCC-2 manual with a sample of elementary school-aged children on the autism spectrum whose teachers completed the measure.

Likely due to the unique gap it fills in the social-pragmatic assessment literature, the CCC-2 has been extensively studied in research. In a validation study of the U.K. version of the CCC-2 (Bishop, 2003), Norbury et al. (2003) found strong interrater reliability between parents and teachers (r = 0.79) and concluded that the U.K. version of the CCC-2 is valid for distinguishing between children with communication impairments from those with typically developing communication skills. This study also validated the use of the SIDI (called the SIDC in the U.K. version) for identifying pragmatic impairments relative to structural language impairments, but did not endorse the measure for diagnosing pragmatic language impairment or autism spectrum disorder (Norbury et al., 2003). Using exploratory factor analytic methods in a sample of elementary school-age children being served for communication, emotional behavioral, reading or learning disorders, Ash and colleagues (2017) examined two pragmatic composite scores drawn from the CCC-2: (1) SIDI and (2) Pragmatic Composite (PC) from the original CCC (Bishop, 1998). They did not find factor structures that fit the data well for either pragmatic composite and concluded that the CCC-2 did not adequately distinguish social-pragmatic symptoms from general language and social-emotional symptoms in their sample. Furthermore, they found sex differences in the factor structure of both pragmatic composites. In a sample of children with and without attention-deficit/hyperactivity disorder (ADHD), Timler (2014) found evidence of convergent validity of the CCC-2 GCC and Pragmatic Composite (PC) with standardized (CELF-4, Semel et al., 2003; TNL, Gillam & Pearson, 2004) and unstandardized language measures (e.g., mean length of utterance in language sample). In addition, Timler’s (2014) results demonstrated strong sensitivity (100) and specificity (85.3) of the GCC for identifying children with language impairment in that sample.

Though the SIDI is viewed as a valid clinical indicator of pragmatic impairments in children, it has not consistently distinguished between diagnostic categories of autism spectrum disorder, SLI, and pragmatic language impairment or its associated DSM-5 category, Social (Pragmatic) Communication Disorder (APA, 2013). For this reason, various pragmatic composites have been explored in research studies but with limited success and little psychometric data reported (for a list of these composites see Table 2 in Ash et al., 2017). For example, in the original CCC (Bishop, 1998), subscales e-j were used as a Pragmatic Composite (PC; M = 60, sd = 18). Though unstandardized, various interpretations of this composite have been applied to the CCC-2 in several studies with mixed success (e.g., Timler, 2014). In a community sample of 54 9–11-year-old children, the PC demonstrated good internal consistency h’(Cronbach’s alpha = 0.88; Leonard et al., 2011). These are the only known psychometric data on this composite and the sample did not include children with autism.

In the present study we aim to examine the psychometric properties of the CCC-2 as applied to children with educational labels of autism and completed by teachers, providing an updated report of the utility of this measure for clinicians and researchers. Our research questions are:

  1. 1.

    Is the CCC-2 a reliable measure of social-pragmatic communication skills in children with autism as reported by teachers? What is the internal consistency of the measure and its subscales for this sample?

  2. 2.

    Is the CCC-2 a valid measure of social-pragmatic communication skills in children with autism as reported by teachers? Does the CCC-2 demonstrate good convergent and discriminant validity for this sample?

  3. 3.

    Using confirmatory factor analysis, and if necessary, exploratory factor analysis, what is the factor structure of the CCC-2 for this sample of students with autism as reported by teachers? Does it have good construct validity indicated by factor analysis? Do results differ by child biological sex or race?

Method

Data in this study were from a larger randomized intervention study funded by the U.S. Department of Education. See Sam et al., 2020 for details of the parent study. All data were collected at the beginning of the school year before any intervention was implemented.

Participants

Participants were 299 elementary school-aged students in 60 schools located in North Carolina between 5 and 13 years old (M = 8.6 years, SD = 1.7) diagnosed with autism spectrum disorder by their education team using state department of education guidelines which align with DSM-5 criteria (APA, 2013). Specific parent study inclusion criteria were that students had a primary or secondary educational diagnosis of autism and qualified for special education services based on state guidelines. Also, a small number of students participated with an eligibility category of developmental delay and a clinical diagnosis of autism.

Parents reported that 27% of the students were female and 51% were not white (see full sample demographics in Table 1). Research staff administered the Leiter-3 (Roid et al., 2013; sample M = 90.0, SD = 18.1, range = 41–143) to assess nonverbal intelligence. Students included in the sample had a range of IEP goals in both separate and inclusive educational settings in 60 North Carolina public schools.

Table 1 Demographic characteristics of child participants (N = 299)
Full size table

There were 242 teachers for the 299 students, so some teachers completed the CCC-2 for 2–3 of their students. Teachers received a packet with the assessments used in our analyses at the same pre-intervention timepoint (plus a number of other student assessments for the parent study). The same teacher completed all of the assessments for each student. The measures are designed as informant-report, therefore no training was necessary beyond the instructions on forms.

Measures

The Children’s Communication Checklist-2, United Stated Edition (CCC-2; Bishop, 2006) is a caregiver rating scale of 4 to 16-year-old children’s communication skills. It consists of 70 items divided into 10 scales. Caregivers rate each item on a 4-point scale by the frequency with which the child demonstrates the communicative behavior described: “0” for less than once a week or never to “3” for several times a day or always. The subscales are titled (a) speech, (b) syntax, (c) semantics, (d) coherence, (e) initiation, (f) scripted language, (g) context, (h) nonverbal communication, (i) social relations, and (j) interests (Scaled Score M = 10, SD = 3). The sum of the first eight subscale scores yields a standardized General Communication Composite (GCC, M = 100, SD = 15), which is indicative of the overall level of disordered communication a child demonstrates compared to norms. GCC scores less than 80 indicate disordered language. The Social Interaction Difference Index (SIDI) is calculated by subtracting the sum of pragmatic and autism subscales (e, h, i, j) from the language subscale sum (a-d). The SIDI is indicative of the level of disordered structural vs. social-pragmatic communication skills a child demonstrates. A negative SIDI score reflects poor pragmatic language skills relative to structural language skills or deficits in pragmatic language that exceed what would be expected from the child’s structural language level.

The Social Skills Improvement System (SSIS; Gresham & Elliott, 2008) is a caregiver-report measure of 3 to 18-year-old children’s social skills, academic engagement, and problem behaviors. The SSIS has strong psychometric properties and standard scores based on a national sample. Subscale internal consistency for the teacher-rated elementary-age form ranges from 0.83 to 0.97 (Gresham et al., 2011). Domain alphas were in the mid to upper 0.90 s (Crosby, 2011). Test–retest reliability for the domains were in the low 0.80 s (Crosby, 2011). Convergent validity was confirmed for relevant domains with the SSRS for the Social Skills domain (Gresham et al., 2011), the Behavior Assessment System for Children for the Problem Behaviors domain (2nd ed.; BASC-2; Reynolds & Kamphaus, 2004). It yields scores in two domains: Social Skills and Problem Behaviors. Social skills are measured in the areas of communication, cooperation, assertion, responsibility, empathy, engagement, and self-control. Though the SSIS has both parent and teacher forms, parents completed the SSIS for this study. The Social Skills standard score was used as a measure of convergent validity and the Problem Behaviors domain standard score as a measure of discriminant validity with the CCC-2.

The Vineland Adaptive Behavior Scales- Second Edition (VABS-2; Sparrow et al., 2005) is a commonly used standardized caregiver report assessments of adaptive behaviors across four domains: Communication, Socialization, Daily Living, and Motor. The VABS-2 has been used in many clinical and research studies with children with autism and has sound psychometric properties. Internal consistency using split-half reliability across the age groups yielded Communication domain correlations from 0.84 to 0.93, Daily Living Skills domain correlations ranged 0.86 to 0.91, Socialization domain correlations from 0.84 to 0.93, and Motor Skills domain correlations from 0.77 to 0.90 (Community-University Partnership for the Study of Children, Youth, and Families [CUP], 2011). The Adaptive Behavior Composite reliability ranged from 0.93 to 0.97 across the age groups (CUP, 2011). Test–retest reliability average correlations were found to range between 0.76 and 0.92 across domains and interrater reliability average correlations ranged between 0.71 and 0.81 across domains/subdomains (CUP, 2011). Parents completed the VABS-2 for the present study. The Communication and Socialization domain standard scores from the VABS-2 were used for convergent validity in this study. The Daily Living domain standard score was used as a measure of discriminant validity.

The Social Communication Questionnaire—Lifetime (SCQ; Rutter et al., 2003) is a standardized caregiver report screening for symptoms of autism with strong psychometric properties. It consists of 40 yes/no questions which provide a total score and subscales. Higher total scores on the SCQ indicate more severe autism symptoms. The Social and Communication subscales of the SCQ were used for convergent validity of the CCC-2 in the present study. The Restricted Repetitive Behaviors subscale was used for discriminant validity. The SCQ was completed by parents of the students in this study.

Participant Demographics were reported by parents of students enrolled in the study. Two of these demographic questions were used in analyses for the present study to address our third research questions. Child biological sex was reported as male or female and analyzed as such. Child race and ethnicity was reported as American Indian, Asian, Black, Hispanic, Multiracial, Other, or White. For our analyses, race and ethnicity was coded as either Non-White/Hispanic/Multiracial or White/Non-Hispanic.

Analytic Approach

Data from the larger study sample (N = 486; see Sam et al., 2020 for further parent study details) were first cleaned for the CCC-2 analysis by excluding participants based on (1) missing critical data (n = 92; e.g., no response to CCC-2 screening questions or CCC-2 not completed) and (2) the CCC-2 screening questions (hearing loss [n = 2], did not speak English fluently [n = 24], did not speak using sentences [n = 58]. Please note that some participants had multiple screening questions checked.). After removing these participants, we examined the sample data and found that the intelligence quotient range on the Leiter-3 still included children with lower non-verbal IQs and adaptive scores than would be expected from children speaking in complete sentences. Though we understand that language, adaptive skills, and cognitive ability in children with autism can be quite variable and that standardized testing does not always capture a child’s skills, we confirmed complete sentence use with teacher report on the VABS-2 (Expressive Communication Subdomain, excluded those with Item 11 score of “0” for “Tells about experiences in simple sentences”) to confirm the reported language level screening from the CCC-2. This additional screening excluded 33 additional children from the sample but yielded a more valid sample for our analysis (N = 299).

Data were first examined for reliability and validity using descriptive statistics for each subscale reported in the manual. Cronbach’s alpha was used to calculate reliability of the subscales and GCC for this sample. By Cronbach’s guidelines, alphas lower than 0.70 have “unacceptable” internal consistency (Cicchetti and Sparrow, 1990; Cronbach, 1951). Pearson correlations were run to determine associations with convergent and discriminant validity measures. Correlation coefficient magnitude was interpreted using Hemphill’s (2003) guidelines where < 0.20 = small/lower third, 0.20–0.30 = medium/middle third, and > 0.30 = large/upper third.

Structural equation models were conducted using Mplus Version 8.4 (Muthén & Muthén, 2017). Multi-level structural equation modeling was used for our confirmatory and exploratory factor analysis models clustered by school to account for teachers completing the CCC-2 for multiple students with autism. First, we conducted multi-level confirmatory factor analyses (CFA) to test to test the fit of the 10 theoretical subscales or three theoretical domains (Structural Language, Pragmatic Language, and Social Communication) of the CCC-2 as described in the manual. CFAs with categorical variables were employed using the mean and variance adjusted weighted least squares estimator (WLSMV; Muthén, 1993). If CFA indicated that these models fit the data well, we would use the model in structural equation models to examine the relationships of these factors with race, biological sex, and cognitive level of children with autism. However, if the data did not fit, our second step would be to move to a multi-level exploratory factor analyses (EFA) to identify and test the best fitting and most parsimonious factor solution. An oblique rotation will be used because oblique rotations allow for the factors to be correlated. We anticipate that factors will be correlated because they are all various aspects of communication and social skills. If the factors were truly uncorrelated, an oblique rotation would yield the same solution as an orthogonal rotation. Therefore, an oblique rotation allows for examination of the optimal solution if factors have a correlation > 0 (Osborne, 2015). In the EFA, the number of potential factors would be examined using (1) scree plot to identify factors that fall below the elbow, (2) factors that have eigenvalues > 1.0, and (3) model fit indices. Model fit would be evaluated using absolute and relative goodness-of fit indices (Hu & Bentler, 1999) including chi-square (x2/d, values less than 2 are indicative of close fit; Hoelter, 1983), the Tucker-Lewis Index (TLI) and the Comparative Fit Index (CFI, values higher than 0.90 and 0.95 suggest adequate and excellent model fit; Bentler & Bonett, 1980), the Root Mean Square Error of Approximation (RMSEA, values < 0.08 and < 0.05 suggest adequate and excellent model fit) and the Standardized Root Mean Square Residual (SMRM, values < 0.10 and 0.08 suggest adequate and excellent model fit (Browne & Cudeck, 1992; Hu & Bentler, 1999; Muthén & Muthén, 2010). In our third and final step, the measurement model in the EFA would be used in in structural equation modeling to examine the relationship between social-pragmatic factors and individual factors (race, biological sex, and non-verbal cognitive level). Since EFA does not allow for covariates in the model, this additional step would be necessary to examine the relationships among individual factors and the factors identified in the EFA. Note that our sample size is limited due to students being nested in classrooms. Thus, even though when using EFA and CFA, it is best practice to split the sample to have a calibration (EFA) and validation (CFA) samples, splitting the present sample would significantly reduce the cluster size, therefore the full sample must be used for both the EFA and CFA. Similar methods have been used in the autism literature (e.g., Čolić & Milačić-Vidojević, 2021; May et al., 2020; Medeiros et al., 2017.

Results

In this school-aged sample of students with autism, the mean GCC Standard Score on the CCC-2 was 77.66 (sd = 13.18, range = 48–118). Subscale scaled score means for this study sample compared to the normative clinical autism sample (Mean age = 8.4 years, sd = 3.4) in the CCC-2 Manual (Bishop, 2006) are in Table 2. The mean SIDI was −1.71 (sd = 9.99, range = −28–29). A score of -10 to 10 on the SIDI is “average.” Scores ≥ 11 indicate SLI (9% this sample), while scores < −10 indicate possible autism (21% of this sample).

Table 2 CCC-2 scores for study sample and normative autism sample from manual
Full size table

Internal Consistency

Cronbach’s alpha was 0.94 for the 70 items in the whole measure. By subscale, Cronbach’s alphas were speech = 0.86, syntax = 0.84, semantics = 0.62, coherence = 0.80, initiation = 0.67, scripted language = 0.64, context = 0.73, nonverbal communication = 0.74, social relations = 0.67, and interests = 0.70. Domain alphas were Language = 0.93, Pragmatics = 0.88, Autism = 0.76. Four of the CCC-2 subscales (Semantics, Initiation, Scripted Language, and Social Relations) had unacceptable alphas (Cicchetti and Sparrow, 1990; Cronbach, 1951).

Validity

For convergent validity, the CCC-2 GCC was highly correlated with the VABS-2 Communication (r = 0.67, p < 0.01) and Socialization (r = 0.61, p < 0.01) domain standard scores. It was also highly correlated with the SSIS Social Skills Standard Score (r = 0.42, p < 0.01). The GCC had a small but significant relationship with the SCQ total score (r = −0.16, p = 0.03), and social subscale (r = −0.19, p < 0.01), but a nonsignificant and small correlation with the communication subscale (r = −0.07, p = 0.32). Note that a negative relationship is expected between the GCC and the SCQ.

The CCC-2 GCC did not show discriminant validity with the VABS-2 Daily Living Skills Domain standard score in that the correlation was significant and the same magnitude as the relationship with the Socialization domain (r = 0.61, p < 0.01). The GCC demonstrated discriminant validity with the SSIS Problem Behavior Standard Score (r = −0.31, p < 0.01) and the SCQ restricted and repetitive behavior subscale (r = −0.08, p = 0.28).

Factor Structure

In the first step, we conducted two 2-level CFAs to test the fit of the 10 theoretical subscales or three theoretical domains (Language, Pragmatics, and Autism) of the CCC-2 as described in the manual. The strengths items (items 51–70) loaded together rather than onto separate factors in both the initial 10-factor and the 3-factor CFAs; therefore, these items were removed from all subsequent models and we tested a 50-item measure. The 10-factor model (with items 51–70 removed) demonstrated acceptable fit, but the 3-factor model did not (see Table 3 for fit statistics of all models tested).

Table 3 Model fit indices
Full size table

In the second step, due to the 3-factor model fitting poorly, a 2-level EFA was performed to identify a more parsimonious model with an oblique rotation. Examination of the eigenvalues suggested that the 11th potential factor was the last factor with an eigenvalue > 1.0 and the scree plot showed the last major “break” was between factors 3 and 4. Fit statistics improved with each additional factor. The 3-factor model was considered the most interpretable solution. The first factor included items concerning structural language (Structural Language Factor). The second and third factors both included items concerning pragmatics, but the second factor included pragmatic communication items (Pragmatic Communication Factor), and the third factor included pragmatic social items (Pragmatic Social Factor). Items with high loadings on multiple factors were examined conceptually before being assigned to a factor or removed. Item factor loadings > 0.35 were considered substantive (Floyd and Widaman, 1995). Four items with factor loadings < 0.35 (3, 4, 21, 41) and eight items with substantive loadings on multiple factors (10, 11, 13, 15, 25, 28, 30, 48) were excluded from the final model.

In the third step, a structural equation model was performed to examine the impact of covariates on the three factors identified in the EFA. The measurement model of the CFA included three factors including the final 38-items of the EFA. The model met acceptable goodness of fit-criteria (RMSEA = 0.050, CFI = 0.896, TLI = 0.889, SRMR = 0.101; see Table 4 for item loadings). Reliability for the three factors was calculated using Cronbach’s alpha: Structural Language = 0.90, Pragmatic Communication = 0.84, Pragmatic Social = 0.76. The Structural Language factor was significantly associated with the Pragmatic Communication (β = 0.38, p < 0.l001) and Pragmatic Social (β = 0.40, p < 0.001) factors. The Pragmatic Communication and Pragmatic Social factors were also significantly correlated (β = 0.41, p < 0.001).

Table 4 CFA item loadings by factor
Full size table

Nonverbal IQ, biological sex, and race/ethnicity were then entered into the measurement model and a structural equation model was used. Nonverbal IQ was significantly associated with Structural Language (β = −0.02, p < 0.001) and Pragmatic Social (β = −0.01, p < 0.001), but not Pragmatic Communication. Individuals with higher nonverbal IQ scores had fewer difficulties with structural language and pragmatic social skills. Race and ethnicity was associated with teacher-reported Structural Language (β = −0.40, p < 0.001), but not Pragmatic Social or Pragmatic Communication. Students who identified as White and non-Hispanic had fewer teacher-reported difficulties with structural language than students who identified as non-White, Hispanic, or multiracial in this sample. No significant differences were found for males vs. females in this sample (Structural Language β = 0.03, p = 0.89; Pragmatic Social β = 0.31, p = 0.06; Pragmatic Communication β = 0.05, p = 0.79).

Discussion

This study examined the psychometric properties of the CCC-2 in a sample of school-aged children with autism with teachers as reporters. Based on alpha coefficients, internal consistency of the whole CCC-2 and domains was good, but the 10 theoretical subscales did not all have sufficient reliability based on alphas. Convergent validity was established between the CCC-2 GCC with the VABS-2-2 Communication and Socialization domain standard scores as well as the SSiS Social Skills Standard Score. The lower magnitude correlations found between the GCC and SCQ Total and Communication scores are to be expected given that the SCQ is an autism-specific diagnostic tool, rather than a general communication measure like the GCC. Discriminant validity of the GCC was established with the SSiS Problem Behavior Standard Score and the SCQ restricted repetitive behavior subscale, but not with the VABS-2 Daily Living Skills Domain standard score. This is likely because the VABS-2 Daily Living Skills Domain is highly correlated with the other domain standard scores (Communication = 0.81, Socialization = 0.67) and overall Adaptive Behavior Composite (0.94) on the VABS-2-2, indicating that the Daily Living Skills domain may be more representative of the student’s overall adaptive skills rather than a distinct domain within those skills which would be expected to show clear discriminant validity with the GCC on the CCC-2.

The EFA with the current sample shows that a 3-factor model is a good fit, although the factors differ from those theorized in the CCC-2 manual and used a smaller subset of items in the final model. The 10 theoretical subscales in the manual demonstrated acceptable fit indices using CFA and the GCC was also psychometrically sound in this sample, which indicates that clinical use of the subscales and GCC with school-aged children with autism is supported by some types of validity. For research purposes, 10 subscales are harder to interpret, thus, the three theoretical domains (Structural Language, Pragmatic Language, and Autism) from the manual were tested using CFA and found to have poor fit indices. Next, EFAs were used to explore factor structures that best fit the data and ultimately led to the 3-factor model. It should be noted that our sample differed from the normative sample used for CCC-2 in that ours consisted entirely of children with autism, our reporters were classroom teachers, and our sample was collected over 10 years after the manual was published following societal growth in understanding of autism (e.g., public awareness of autism, increased diagnosis of girls/women) as well as published changes to the DSM-5 (APA, 2013).

Though subscale sample means were all below average according to test norms, this sample of students with autism scored slightly higher on all subscales and composite scores than the autism student sample in the CCC-2 manual from 2006. For example, the mean GCC score for the normative autism sample was 72.4, which is 5 points lower than the present sample. Moreover, over 27% of the normative autism sample would meet cut-off for possible autism on the CCC-2 while 21% of the present sample would. This slight reduction in cut-off for autism is likely due to the change to DSM-5 which includes children who were formerly diagnosed with Asperger syndrome and PDD-NOS as part of autism spectrum disorder. Factors such as increased autism awareness, lower age of diagnosis, and increased access to evidence-based treatments for students with autism may also contribute to the increase in these scores. For instance, the average age of autism diagnosis when the CCC-2 was published in 2006 was 50–60 months (Rice et al., 2009), while the mean age of diagnosis for our sample was 3.96 years or approximately 48 months (sd = 1.84 years, range 1–10 years).

A 3-factor model best fit the CCC-2 items for this sample of children with an educational diagnosis of autism (see Table 4 for item loadings on each factor). The first factor, which we labeled Structural Language, includes items related to syntax (e.g., mixing pronouns), semantics (e.g., mixing up words with similar meanings), speech (e.g., mispronouncing words), coherence (e.g., confusing the sequence of events when telling about an event), and comprehension (e.g., misinterpreting what has been said). This first factor had the most alignment with the CCC-2 manual and was the more straight-forward to interpret. Evidence-based instructional practices that may benefit these students include Prompting, Reinforcement, Modeling, Visual Supports (Hume et al., 2021) and individual or group speech-language therapy.

The second factor, which we called Pragmatic Communication, consists of items related to sharing about high interest topics (e.g., talks about lists of things they find interesting, shows interest in things others may find unusual, moves the conversation to a preferred topic), using scripted or repetitive language, and lack of awareness of social expectations in conversation (e.g., telling people things they already know, forgetting to give background about a topic, difficulty taking turns in conversation, standing too close to people). The Pragmatic Communication factor seems to capture the pragmatic challenges of students with autism who are socially engaging in conversation with others, but do not always meet the conversational expectations of neurotypical communication partners. Clinically, students with low scores on this factor, may need support to understand the social expectations of neurotypical people and how the mental states of others may differ from their own. We also recommend peer education so that non-autistic students understand the pragmatic communication style of the student with autism. Evidence-based instructional practices that may benefit these students include Social Skills Training, Visual Supports, Social Narratives, and Peer Mediated Instruction and Intervention (Hume et al., 2021).

The third factor labeled Pragmatic Social includes items related to social engagement with others (e.g., lack of response to conversational overtures from others, lack of eye contact, lack of facial expression, seems distant and preoccupied even with familiar adults). We interpreted this as the profile of students who struggle with engaging in social interactions, and the non-verbal aspects of social engagement. Clinically, these students may need support to initiate and maintain interactions with others. They may also benefit from peer and school-wide education in order to consistently positively reinforce social engagement and to give them space to process or take breaks from social interactions when needed. Evidence-based instructional practices that may benefit these students include Prompting, Video Modeling, Visual Supports, Social Narratives, Peer Mediated Instruction and Intervention, and Reinforcement (Hume et al., 2021).

We examined these three factors for differences by nonverbal IQ, biological sex, race and ethnicity. The association found between nonverbal IQ and the Structural Language and Pragmatic Social, but not the Pragmatic Communication, factors is aligned with our interpretation of the Pragmatic Communication factor as indicative of more socially engaged students with autism. Student race and ethnicity were significantly associated with Structural Language (B = −0.40, p < 0.001), but not Pragmatic Social or Pragmatic Communication. Students who identified as White and non-Hispanic had fewer difficulties with Structural Language than students who identified as White and Hispanic/Latino, or multiracial in this sample. This likely reflects a racial/ethnic bias in the CCC-2 items, and/or the teachers completing the CCC-2. Items in the Structural Language factor are based on White/Non-Hispanic expectations of grammar, sentence structure, and vocabulary that may not be culturally sensitive. For example, African American English (AAE) has documented lexical, phonological, and grammatical differences from other American English dialects (Charity, 2008) and these dialectical differences may be incorrectly counted as deficits on this factor of the CCC-2. This bias should be considered when asking classroom teachers to complete the CCC-2, and we recommend observation and assessment of the student with a culturally sensitive lens to determine if the student has a true language disorder versus a language difference (see Bland-Stewart, 2005).

Limitations

There were limitations to this study. First, schools in the larger study from which this sample was drawn were all from one U.S. state, making results less generalizable. Furthermore, data on the CCC-2 were teacher-reported and were therefore limited to their interpretation of the items and subject to potential biases. For instance, a parent/caregiver report may be more sensitive to the student’s cultural/linguistic background when reporting on aspects of social communication, while a teacher’s report may be based on traditional American classroom and student expectations. As these were extant data from a larger funded study, we did not have test–retest reliability or inter-rater reliability to report for the CCC-2. Finally, while our sample was large for an autism trial, its size limited our analytic abilities due to the students being nested in classrooms. Though, when using EFA and CFA, it is best practice to split the sample to have a calibration (EFA) and validation (CFA) samples, splitting the present sample significantly reduced the cluster size, therefore the full sample had to be used for both the EFA and CFA. Similar methods have been used in the autism literature (e.g., Čolić & Milačić-Vidojević, 2021; May et al., 2020; Medeiros et al., 2017); however, future psychometric studies are needed with samples representative of the national population of students with autism and with samples large enough to run more rigorous analyses.

Recommendations and Conclusions

Based on the results of this analysis, we have several recommendations for using the CCC-2 for teacher-reported data of students with autism. First, for educators and clinicians, we recommend using the GCC as an overall communication score for elementary school-aged students with autism because it had good psychometric properties. When helpful, we recommend using the 10 theoretical subscales and three domain scores for this population in educational and clinical settings as well. Second, for school-based research purposes with samples similar to the one presented in this study, we recommend only administering all items on the CCC-2 but limiting interpretation to the 38 items included in our final factors because other items did not load well during factor analysis and may not be relevant to students with autism or teacher-reporters. However, researchers may want to conduct a content analysis to make sure eliminating items does not lose any data essential to their research questions. Third, for school-based research using school samples similar to those described in the present study, using the three factors found in our factor analysis will provide the most valid data for summary scores of Structural Language, Pragmatic Communication, and Pragmatic Social. We recommend using these factors rather than the Social Interaction Difference Index (SIDI) or three theoretical categories in the CCC-2 manual when describing a student’s teacher-reported communication strengths and weaknesses on the CCC-2, particularly when describing social-pragmatic vs. structural language skills. Fourth, we do not recommend use of the Autism cut-offs from the CCC-2 as reported by teachers to qualify students for autism services, as this would miss 80% of students with autism in our sample. Please note that qualifying students for services was never a purpose of the CCC-2 as described in the manual; our analysis simply supports this point. Finally, we suggest caution in interpreting Structural Language results from non-White, non-Hispanic students due to possible item or reporting biases.

In conclusion, the CCC-2 is a valuable tool for gaining teacher report of student communication and social-pragmatic skills in the school setting. Our findings offer psychometric support specific to this setting and population that are not available in the manual.