Introduction

Spoken and nonverbal social communication skills are a core challenge for children with autism spectrum disorder (ASD), with approximately 30% considered minimally verbal with less than 20 spontaneous spoken words at school entry (Tager-Flusberg and Kasari 2013). Social communication (Mundy et al. 1987) includes both requests (communication to have needs met), and joint attention (communication to socially share interest including spoken comments). Although intervention for beginning communicators often focus on developing requesting contingencies, initiating spontaneous communication to share is a core challenge that uniquely distinguishes children with ASD from children with other developmental disorders (Mundy et al. 1987). These impairments are compounded in older children who remain minimally verbal. However, recently interventions have reported improvements for school age children who are minimally verbal in nonverbal and verbal joint attention skills (Almirall et al. 2016), and spontaneous spoken communication (Kasari et al. 2014a) in semi structured assessments with independent evaluators.

The inclusion of caregivers in these interventions has the potential to generalize clinical interventions and to enrich the daily interactions between caregivers and their children. For example, RCTs have demonstrated that caregivers can increase the rate and quality of opportunities for their preschool aged children with ASD to engage in a shared activity (e.g., Green et al. 2010) and initiate joint attention skills (Kasari et al. 2014b). Specifically, with children who are minimally verbal, caregivers made significant gains in their implementation of intervention strategies, and their implementation was associated with children spending more time jointly engaged in play-based interactions (Shire et al. 2015). However, for children who are minimally verbal, there is less information on the relation between caregivers’ strategies and children’s spoken language outcomes.

When transferring intervention strategies to others including caregivers, published research aims to transfer all of the strategies to the person who is learning the intervention. The caregiver mediated intervention literature varies in whether and how caregiver strategy use is measured and then associated with children’s outcomes (Stahmer and Pellecchia 2015). When outside observation of caregivers’ strategies is applied, mixed findings are reporting ranging from no association between observer rated caregiver strategy use and child outcomes (e.g., Rogers et al. 2012), to concurrent changes in observed caregiver strategies and child outcomes (e.g., Kasari et al. 2014b).

When examining implementation fidelity, 80% is a common marker for successful delivery (e.g., Wilczynski and Christina 2008). However, neither attainment of this marker or any other has been empirically tested within specific intervention packages. For most interventions, it is not known how well a package of strategies must be delivered to be considered high quality nor what level of implementation is required in order to see change in children’s corresponding outcomes. It also possible that lower level implementation may lead to change in children’s outcomes that may be more malleable and not central to the core diagnostic criteria. For example, the delayed development of spontaneous initiations of joint attention is a unique diagnostic marker of autism, while delays in requesting skills are not as pronounced nor specific to autism (Mundy et al. 1986; Jones and Carr 2004). Therefore, it is not clear what degree of intervention implementation is needed to effectively support gains in core social communication challenges and in particular, for initiating joint attention language (herein referred to as comments). Therefore, this paper aims: (a) to examine the association between caregivers’ intervention implementation with children’s spoken language (spontaneous communicative utterances including requests and comments) and more specifically, spontaneous comments, (b) to examine the degree to which caregivers must demonstrate strategy mastery in order to anticipate gains in children’s spontaneous comments, as well as (c) to unpack caregivers’ total implementation scores by describing and exploring the strategy subscales embedded within the total score.

Methods

Participants

Children

Twenty-two children age 5–8 (mean age = 6.74 years; SD = 1.16) who were included in a larger multi-site intervention trial were included from the site where caregiver–child interactions were captured monthly. Children were male, demonstrated less than 20 spontaneous words on a natural language sample, evidenced a developmental age of at least 24 months (across two of three language and cognitive assessments), and were diagnosed with autism (confirmed with the Autism Diagnostic Observation Schedule-Generic (ADOS-G) Module 1 by research reliable assessors; Lord et al. 2000). Children must have also received at least 2 years of intervention services (e.g., early intervention, speech language services, school based services) to be included in this study. Caregivers reported the child’s intervention history at screening including predominantly applied behavior analytic services (n = 21) and speech (n = 22) as well as occupational therapy, Floor Time, Music, and adaptive physical education. An average of 3.91 services were reported across families with services starting between 2 and 3.5 years of age. Parents reported children’s race as African American (n = 2), Asian (n = 5), Caucasian (n = 12), Hispanic (n = 2), and multiethnic (n = 1). Children entered with receptive language at 2.38 years (SD = 0.76), expressive language at 1.83 years (SD = 0.48) as measured by the Test of Early Language Development (TELD-3: Hresko et al. 1999), and nonverbal cognitive scores of 4.26 years (SD = 1.13) measured by the Leiter International Performance Scale—Revised: Roid and Miller (2011).

Caregivers

Sixteen mothers, and six fathers were the primary caregiver participants. All caregivers had completed at least some college, nine completed a college degree, and six obtained graduate degrees. Two families withdrew at midpoint for reasons unrelated to the intervention (e.g., moved away, job demands) but returned for assessments at study exit; therefore, their data are included in the analyses.

Randomization, Design, and Intervention

Participants were initially randomized to JASPER-EMT intervention alone or JASPER-EMT plus access to a speech generating device (SGD) in an adaptive treatment design (see Kasari et al. 2014a). The JASPER-EMT intervention includes a combination of joint attention symbolic play engagement, and regulation (JASPER: Kasari et al. 2014a) and enhanced Milieu teaching (EMT: Kaiser et al. 2000). A naturalistic developmental behavioural intervention, sessions occurred at a university clinic and took place in the context of play. Design and intervention details are published in Kasari et al. (2014a). The social communication intervention was 6 months long with parents observing for the first 3 months and receiving active in session coaching for the last 3 months (see details in Shire et al. 2015).

Measures

Caregiver–Child Interaction (CCX: Adapted from Kasari et al. 2010)

The CCX includes a standard set of toys (including a ball, blocks, vehicles, dinosaurs, dishes with food, dolls and furniture, drum with sticks, a puzzle and a shape sorter). Families were asked to play with their child as they usually would for 10 min. The CCX was collected eight times including monthly intervals from entry to exit, and at 3-month follow up. The CCX videos were coded for children’s spontaneous language (spontaneous communicative utterances and comments) as well as caregivers’ strategy implementation.

Children’s Spontaneous Communicative Utterances (SCU) and Comments

The CCX was transcribed and coded for function (request, comment, other-non social) and for spontaneity (spontaneous initiation, prompted, or imitated utterance) from the CCX. Both spoken and augmented (SGD) utterances were included. Counts of spontaneous requests and comments were summed to obtain total SCU. Spontaneous comments included non-elicited, non-imitated, spoken or augmented words for the purpose of socially sharing (joint attention). For example, the child and parent are building a block tower that falls over. The child says “crash!” to share the event with their caregiver. Coders were instructed to attend to secondary indicators to confirm if the utterance was a comment which could include eye contact, body orientation toward the caregiver, evidence of joint engagement in the activity (e.g., active turns), and the whether the utterance matches the context of the interaction (e.g., commenting crash when the blocks fall over versus an unrelated script such as counting). Transcribers and coders were blinded to time point. 20% of the videos were double coded for reliability. Intraclass correlations (ICC: two way mixed model) were calculated for spontaneous (comments: 0.98; requests: 0.93; other: 0.99, SCU: 0.97), imitated (comments: 0.99; requests: 0.96; other: 0.99), and prompted language (comments: 0.98; requests: 0.99; other: 0.98).

Caregivers’ Strategy Implementation

Caregivers’ total strategy implementation score (percentage) was obtained by summing scores across 53 items each rated from 0 to 5 and dividing it by the total number of possible points (see previous publication Shire et al. 2015). These items covered eight strategy subscales including basic interaction strategies, environment, imitation and modeling, language, establishing play routines, expanding play routines, programming for joint attention and requesting, as well as prompting and requesting strategies (time delays and EMT requesting episodes). Descriptions of the subscales can be found in Appendix. Two independent raters scored total strategy implementation (ICC = 0.997).

Statistical Analysis

Primary

A generalized linear mixed model with a negative binomial distribution was applied to each outcome: children’s SCU (counts) and comments (counts). Time, treatment condition (spoken or spoken plus SGD), and time by treatment condition were included as parameters in the model while controlling for children’s entry non-verbal IQ (Leiter-R). Caregivers’ total strategy implementation score was also included as time varying covariate.

Secondary

To examine the degree to which caregivers’ must appropriately apply the strategies to see an increase in children’s spontaneous comments, caregivers’ strategy use was examined in a negative binomial regression model. The analyses explore possible binary cut points for caregivers’ strategy adoption at study exit (e.g., above or below a total implementation percentage score). The assumptions for all final models were carefully evaluated (i.e. there was no evidence of over and under dispersion for counts data).

Exploratory

Individual JASPER strategy subscales were analyzed using descriptive statistics only as an exploratory analysis.

Results

Descriptive Statistics

Treatment assignment was not a significant predictor within any of the following models. Therefore, values are provided for the groups combined. Children demonstrated an average SCU of 4.38 (SD = 5.16, min = 0, max = 17), and 1.19 (SD = 2.4, min = 0, max = 11) comments at entry. At exit, children averaged an SCU of 12.35 (SD = 9.74, min = 3, max = 47), and 5.5 (SD = 9.19, min = 0, max = 39) comments. Descriptive statistics for all eight time points are provided in Table 1. Gains from entry to exit in caregivers’ implementation are statistically significant as previously reported (Shire et al. 2015). The total implementation score was made up of eight subscales including basic strategies (exit M = 69%, SD = 13%), environment (M = 80%, SD = 15%), imitation and modeling (M = 77%, SD = 14%), language (M = 75%, SD = 16%), play routines (M = 74%, SD = 18%), expanding play (M = 31%, SD = 38%), programming communication (M = 62%, SD = 21%), and prompting (M = 88%, SD = 19%).

Table 1 Descriptive statistics for children’s communication: entry to exit
Full size table

Primary: Caregivers’ Implementation and Children’s SCU

Gains in children’s total SCU from entry to follow up were significant [f(1,129) = 6.06, p = .015]. There was no significant interaction between time and treatment condition [f(1,129) = 1.21, p = .273]. Caregivers’ implementation was included as a time varying covariate where a significant association was found between caregivers’ implementation and increases in children’s SCU [f(1,129) = 6.91, p = .01].

Primary: Caregivers’ Implementation and Children’s Comments

Average gains in comments alone from entry to follow up were modestly significant [f(1,129) = 3.50, p = .064]. There was no significant interaction between time and treatment condition [f(1,129) = 2.12, p = .148]. However, looking further at commenting, caregivers’ strategy implementation was examined as a time varying covariate which significantly predicted children’s spontaneous comments [f(1,129) = 17.25, p < .001].

Secondary: Exploring Degree of Caregivers’ Implementation

Although group gains in comments were not significant, considering the variability in gains in children’s comments (exit range 0–39), further examination of caregivers’ support of commenting was made. In a preliminary effort to examine the minimum total strategy implementation required to support children’s gains in comments, two benchmarks of above or below 70 or 75% (binary predictor) were selected to represent first average strategy implementation at exit (70%) and a small increase over average caregiver implementation at study exit (75%), but less than the standard of 80% typically required of clinicians.

Demonstration at least 70% intervention implementation at exit was a significant predictor of children’s comments at exit (χ2 = 6.43, p = .011). Further, reaching at least 75% implementation at exit was also a significant predictor of children’s comments at exit (χ2 = 5.98, p = .015) after adjusting for treatment group differences.

Eleven families reached 70% implementation at exit. For those who met 70% implementation, significant growth in children’s commenting was found from study entry to exit [f(1, 130) = 13.04, p < .001]. However, for families who were below 70%, children did not show significant gains in commenting [f(1, 130) = 3.23, p = .075].

When the benchmark is raised to 75%, 10 families reached this level of implementation by exit. For those caregivers who demonstrated at least 75% implementation, their children also showed significant growth in commenting [f(1, 130) = 11.36, p < .001]. However, for families who delivered the strategies at less than 75%, children showed modestly significant gains in comments [f(1,130) = 3.69, p = .06].

Children’s growth in commenting differs above and below 70% caregiver implementation. However, these differences in children’s comments lessen (p = .06) when the benchmark for implementation is set at 75%. Seeing as gains in comments for those children with caregivers below 75% implementation are not reaching a p value of .05, total intervention implementation at 75% may be considered a conservative estimate of the minimum strategy implementation required to advance commenting skills (Fig. 1).

Fig. 1
figure 1

Gains in children’s spontaneous comments from entry to exit in dyads with caregiver implementation caregivers above and below 75%

Full size image

Exploratory: Descriptive Subscales

Due to the differential finding where only 75% implementation was associated with children’s commenting while both 70 and 75% implementation were significantly associated with children’s SCU, we explored descriptive data from the eight JASPER strategy subscales that make up the total implementation score. Figure 2 depicts the mean subscale scores for caregivers above 75% total implementation and those below. The greatest differences in means subscale score between caregivers demonstrating 75% or greater implementation and those with total scores below 75% were in establishing play routines (< 75% M: 48.6%; 75%+ M: 84%), expanding routines (< 75% M: 17.9%; 75%+ M: 44.53%), and programming for joint attention and requesting (< 75% M: 45.0%; 75%+ M: 78.8%).

Fig. 2
figure 2

Children’s spontaneous language by JASPER strategy subscale at exit: caregiver implementation above and below 75% per subscale

Full size image

Discussion

This study focused specifically on examining the statistical association between caregivers’ intervention strategy use and children’s spontaneous language. Consistent with prior JASPER intervention studies where therapists and caregivers could significantly increase children’s time jointly engaged in play, the current study extends these findings to demonstrate that caregivers’ strategy implementation when examined as a time varying covariate was associated with their children’s spontaneous communication (SCU). SCU is a combination of child initiated requests and comments, indicating that gains in children’s SCU also generalized beyond interactions with clinical staff to interactions with caregivers. For children with complex communication needs, providing caregivers with access to strategies delivered in clinic-based intervention can support children’s spontaneous communication in novel, unsupported play based interactions with their caregivers.

Although 80% is a common yardstick of implementation and reliability, empirical data linking this or any implementation standard has not been developed for JASPER clinicians or community stakeholders. Considering that caregivers’ average intervention implementation at exit was approximately 70%, we explored 70% as a benchmark for implementation and then a small rise above average at 75%. When families demonstrated at least 70% implementation, children’s gains in comments were significant. However, below 70% implementation children’s gains in comments were not significant and only modestly significant below 75% implementation. This preliminary finding suggests that a minimum of 70% intervention implementation may support gains in the core challenge commenting for children with autism. Although gains in comments were modestly significant at 75%, this finding should be interpreted with caution. Due to the small sample size, it is possible that the study was underpowered to detect the effect, alternatively, growth in commenting is more limited when implementation is below 75%. Further examination of these benchmarks in additional trials that expand the heterogeneity of the sample (e.g., age, language level, intervention agent) may provide additional information and increase the generalizability of the finding.

There are eight subscales that make up a caregivers’ total implementation scores. When exploring the average subscale values for families who achieved at least 75% fidelity and for those who scored below 75%, it is noticeable that as illustrated in Fig. 2, caregivers who demonstrated 75% or greater total implementation, demonstrated the greatest differences in mean subscale scores in: (1) programming for joint attention and requesting, (2) establishing play routines, and (3) expanding play routine strategies. Programming for joint attention and requesting requires that caregivers go beyond responding to children’s communication, and modeling appropriate communication, but to also provide explicit opportunities for the child to comment (e.g., inserting a novel, unexpected, or goofy step into a routine) or request (e.g., playfully hold the last piece of the wooden cake out of the child’s reach). It is logical that providing additional teaching opportunities for social communication may lead to greater gains for children’s development however, high quality implementation of play strategies may also have benefits. Within this intervention, caregivers were taught to establish and expand play routines as the context to converse with their children. Routines allow for sharing of ideas through comments as well as requesting materials. The ability to maintain and then extend the time the child engaged in play routines is challenging but may allow for more opportunities to respond and expand children’s communication, as well as model appropriate social communication skills. Further exploration of these more nuanced indicators of the quality of the social interaction as well as the influence of play strategies may help identify specific supports to foster spontaneous communication.

Limitations and Next Steps

This study included preliminary exploration of the minimum total fidelity score associated with children’s spontaneous communication. With a larger sample size, the analyses could be first applied to explore the stability of the association between the conservative 75% benchmark and children’s commenting language found with this sample. Second, the analyses could be expanded to empirically test specific groups of related strategies (e.g., establishing play routines) or individual strategies to better understand exactly which strategies are necessary to support gains in children’s spontaneous commenting. It is possible that making small gains in specific strategy subscales may have greater impact than others on children’s spontaneous communication.

Conclusions

By learning specific communication strategies delivered in the context of play, caregivers can support the generalization of their children’s gains in treatment to their interactions where the interventionist is not present. Delivery of the intervention package with at least 70% accuracy may support gains in children’s spontaneous commenting, a core challenge for children with ASD.