Introduction

Children with autism spectrum disorder (ASD) exhibit pervasive deficits in social communication and demonstrate restricted and repetitive interests and attention. Deficits in social communication skills, including social engagement, language, imitation, and play, are the earliest identifiable feature of the disorder (e.g., Ingersoll 2011), and are strongly associated with long-term outcomes (e.g., Gillberg and Steffenburg 1987; Howlin et al. 2004). Thus, there is a significant need for interventions that can target these skills early in development (Ingersoll 2011).

One approach to teaching social communication to young children with ASD has been the use of parent–mediated interventions. These interventions, most of which are drawn from developmental and/or naturalistic behavioral theory, teach parents to use naturalistic intervention techniques to promote their child’s social communication skills during play and other daily routines. Parents can use the techniques throughout their child’s day, thereby increasing the intensity of intervention and promoting generalization (e.g., Brookman-Frazee et al. 2009; Koegel et al. 1996). A number of such interventions have been developed (e.g., Aldred et al. 2008; Ingersoll and Dvortcsak 2010; Mahoney and Perales 2003) and there is growing evidence that they lead to improvement in children’s social communication development, including social engagement (e.g., Casenhiser and Shanker 2011 ; Kasari et al. 2010), language (Coolican et al. 2010; Vismara et al. 2009), imitation (Ingersoll and Gergans 2007), and play (Gillett and LeBlanc 2007). However, there have also been several recent RCTs of parent-mediated social communication interventions showing modest or null effects (Carter et al. 2011; Oosterling et al. 2010; Rogers et al. 2012; Siller et al. 2013). For example, a recent RCT of the Hanen More Than Words program (Sussman 1999) found a modest effect of the intervention on child communication, but only for children whose pre-treatment level of object interaction was low (Carter et al. 2011). An RCT of the Focus Parent Training program with 75 toddlers with ASD conducted in the Netherlands found no effects of the intervention on any of the child social communication outcomes (Oosterling et al. 2010).

Parent-mediated interventions are by definition multi-level; they involve the transfer of an intervention from the trainer to the parent and the parent to the child. A failure to find a treatment effect could be due the use of an ineffective intervention with the child or an ineffective intervention with the parent (Wainer and Ingersoll, in revision). Thus, it is important to evaluate the effect of the program on both parent and child behavior. In addition, when a parent-mediated intervention is found to be effective, it is often unclear which elements of the program are responsible for producing the effects. It is possible that changes in parent behavior other than the use of the intervention techniques are responsible for child improvements. For example, parents may begin to spend more time interacting with their child once they begin training, or may begin attending more to specific behaviors in their child that are the target of the intervention. Further, as most social communication interventions for young children with ASD contain multiple techniques, it is often unclear which techniques most influence child behavior. Thus, a better understanding of the active ingredients of promising parent-mediated interventions is needed.

Project ImPACT (Improving parents as communication teachers) is an evidence-based parent training curriculum that teaches parents to promote their child’s social-communication skills during play and daily routines (Ingersoll and Dvortcsak 2010). It utilizes a blend of developmental and naturalistic behavioral techniques to promote social engagement, language, social imitation, and play. Project ImPACT was developed using an iterative process with parents, teachers, and other service providers in order to be compatible with community-based service delivery models. The resulting curriculum can be implemented in either a group or individual setting. Initial evaluations of Project ImPACT have examined the feasibility of the group training model in public EI/ESCS settings (Ingersoll and Dvortcsak 2006; Ingersoll and Wainer 2011). Findings from both teacher reports (Ingersoll and Dvortcsak 2006) and standardized observations of parent–child interactions (Ingersoll and Wainer 2011) provide evidence that parents improved their use of the intervention techniques as a result of the parent training program. Further, the children exhibited gains in language use during a home observation and on parent- and teacher-report measures (Ingersoll and Wainer 2011).

These data are promising. However, the pre-post nature of the design precludes strong interpretations regarding the effect of the training program on parent or child behavior. Thus, the goal of this study was to evaluate the efficacy of the individual parent training model on parent fidelity and child language using a single-subject, multiple-baseline design. In addition, we examined the degree to which parent use of the intervention techniques predicted child spontaneous language in an attempt to identify active ingredients of the program. Finally, in order to understand how the parent training model can be adapted by community programs, we evaluated parent learning during both the original twice per week training model (five dyads) and a modified once per week training model (three dyads).

Methods

Participants

Eight young children (44–80 months old, M = 53 months) with ASD and their mothers participated in the current study. All children met DSM-IV criteria for autism or pervasive developmental disorder-not otherwise specified, as well as the cutoff for autism spectrum disorder on the Autism Diagnostic Observation Schedule (ADOS-G; Lord et al. 2000). Child participants were administered the Bayley Scales of Infant Development, 3rd Ed. (Bayley 2005) to determine cognitive age (M = 25.9 months, SD = 6.5 months) and the Preschool Language Scale (PLS-IV, Zimmerman et al. 2002) to determine language age (M = 22.9 months, SD = 6.0 months). See Table 1 for participant information. This study was approved by the Michigan State University Institutional Review Board. Each parent participant provided their informed consent to be in the study.

Table 1 Participant characteristics
Full size table

Settings and Materials

The majority of baseline and treatment sessions were conducted in a treatment room at an ASD research laboratory. Several pairs of developmentally appropriate toys were provided for each session. An average of 2 (range = 1–3) baseline sessions and an average of 3 (range = 2–4) treatment sessions were conducted in the families’ home with their own toys to assess skill generalization. All sessions were video recorded for later data analysis.

Procedure

An IRB-approved single-subject, multiple-baseline design was conducted across participants (Hersen and Barlow 1976). Participants attended the research laboratory 1 (n = 3) or 2 (n = 5) days a week during baseline and treatment. Dyads were randomly assigned to pre-determined baseline periods (Edgington 1996) and then received 12 weeks of training in Project ImPACT. All parent–child dyads returned to the research laboratory for a 1-month follow-up session to determine maintenance of skill and to update parent and child goals.

Baseline

During the baseline phase, mothers were provided with developmentally appropriate toys and were asked to play with their child as they usually did for 10 min.

Treatment

Participating families worked with their own parent trainer during hour-long sessions once or twice per week throughout the study. One of the trainers had a masters degree in social work and the other two trainers were graduate students in a clinical psychology doctoral program. At the beginning of treatment, the parent was provided with a parent manual that presented information about each Project ImPACT intervention technique and included homework and reflection questions. During the first treatment session, the parent and trainer engaged in collaborative goal setting to identify parent-selected treatment targets in the areas of social engagement, language, imitation, and play. Subsequent sessions began with a review of the parent’s homework from the prior session. The trainer then presented a new intervention technique(s), described the rationale for the technique(s), discussed how the technique(s) could be used to address the child’s specific goals, and answered any parent questions. Next, the trainer modeled the technique(s) with the child while the mother watched. The mother then practiced the technique(s) while the trainer provided positive and corrective feedback. At the end of each session, the parent and the trainer developed a homework plan for the parent to carry out between sessions.

The Project ImPACT techniques are taught to parents in a prescribed order because the initial techniques set the foundation for the techniques taught later on. See Supplementary Appendix A for an overview of the intervention techniques introduced during each parent training session. A complete description of the intervention techniques and parent training protocol can be found in Ingersoll and Dvortcsak (2010).

Dependent Measures

The final 10 min of each session were videotaped for later scoring by trained observers blind to the participants’ point in treatment. The parent trainer was not present and did not provide coaching or feedback during these probes.

Parent Fidelity

Parent use of the five fidelity dimensions was rated on a scale of 1 (parent does not implement during the session) to 5 (parent implements throughout the session) using the Project ImPACT fidelity form (Ingersoll and Dvortcsak 2010). A rating of 4 or above (80 %) is considered implementing the intervention strategy with fidelity. An average fidelity score was calculated for each session by averaging parent scores across the five fidelity dimensions. See Table 2 for behavioral definitions.

Table 2 Behavioral definitions
Full size table

Child Spontaneous Language

The children’s spontaneous use of language targets were scored using frequency counts. To be scored as spontaneous language, language targets had to be in context, directed to the parent, and at least 3 s had to occur between therapist’s last utterance and child’s production. A rate per minute of spontaneous language for each session was calculated by dividing the frequency of the child’s use of his or her language targets by the length of the observation. See Table 1 for individual child language targets.

Goal Achievement

Parent-selected goals were developed at the beginning of the treatment phase. Parents collaborated with the trainer to identify specific language targets for the child to work towards during the treatment phase. If a child achieved a particular goal by the end of treatment, a new parent-selected goal was developed. A goal achievement score was calculated by dividing the number of new language goals written at the end of treatment by the total number of initial language goals.

Inter-Observer Reliability

Inter-observer reliability was obtained for 25 % of the sessions by trained observers. Intraclass correlations were used to calculate reliability for fidelity of implementation on Makes Play Interactive (.63), Models and Expands Language (.72), Provides Opportunities for Initiations (.79), Helps Increase the Complexity of Initiations (.77), and Paces the Interaction (.73) and for child spontaneous language (.93).

Data Analysis

Visual inspection of session data was used to examine changes in parent fidelity ratings and rates of child spontaneous language from baseline to treatment, and at follow-up for each participant (Gliner et al. 2000). Additionally, because these data have a two-level nested structure (sessions nested within participants) multilevel modeling was used to examine the effect of treatment phase on outcomes, aggregating across participants (Van den Noortgate and Onghena 2008). This analysis models the variance due to repeated measurements of the outcome for each participant (i.e., variation within participants) as well as the variance due average differences from participant to participant (i.e., variance between participants). For these analyses, missing data due to recording equipment failure (<5 % of sessions) were imputed by averaging the data point immediately preceding and immediately following the missing data point.

The relationship between the parents’ use of the intervention techniques and their children’s rate of spontaneous language was also examined using multilevel modeling. This approach has the potential to identify the active treatment component(s) of an intervention package, by examining which, if any, of its individual components are related to outcomes, after controlling for both within and between participant variance on the outcome. For the first analysis, parent average fidelity was entered as a predictor to examine the relationship between overall parent fidelity and child spontaneous language use. For the second analysis, parent use of the first four fidelity dimensions were entered simultaneously to determine which fidelity dimensions were unique predictors of rate of child spontaneous language use when controlling for the other dimensions. For this analysis, the fifth fidelity dimension, Paces the Interaction, was not included as it represented the parents’ ability to use the other dimensions together, and was highly correlated with the other fidelity dimensions.

Results

Parent Fidelity

Parents were taught the intervention techniques in a phased approach to support parent learning. Each of the intervention techniques taught maps onto a specific dimension of parent fidelity; thus, we examined the parents’ use of each fidelity dimension as it was introduced, as well as the parents’ average fidelity across all of the dimensions.

Parents were rated as having low to moderate fidelity for each of the fidelity dimensions during baseline (see Fig. 1). Use of all fidelity dimensions was relatively stable, with the exception that Parents 1 and 3 showed an increase in Makes Play Interactive during the last baseline session. Despite occasional correct use of the intervention techniques during baseline (resulting in ratings for individual fidelity dimensions ≥4), no parent met overall average fidelity of implementation during baseline, and average fidelity ratings remained relatively low (see Fig. 2).

Fig. 1
figure 1

Ratings of parent fidelity by dimension

Full size image
Fig. 2
figure 2

Average parent fidelity and rate of child spontaneous language

Full size image

With the onset of treatment, all parents began to increase their correct implementation of the intervention strategies. Parents increased their fidelity of implementation for a particular dimension as the corresponding techniques were introduced in session. Although there was some variability, parents generally increased their use of Makes Play Interactive first, followed by Models and Expands Language, then Creates Opportunities for Initiations, Helps Increase the Complexity of Initiations, and finally Paces the Interaction.

An analysis of the parents’ average fidelity ratings, demonstrates that parents showed an increasing trend in their average fidelity scores with the onset of treatment. Parents who received training twice per week required an average of 14 sessions (range = 8–17) before achieving fidelity of implementation (average fidelity rating ≥4). Parents who received training once per week required an average of 6 sessions to achieve fidelity (range = 5–7). All parents achieved fidelity of implementation at some point during the treatment phase; however, after reaching threshold, all parents demonstrated decreases in their average fidelity as additional intervention techniques were introduced. Despite these periodic “dips” in fidelity, all parents maintained higher than baseline rates of fidelity throughout treatment. With the exception of parent 6 who demonstrated a drop in fidelity during the final generalization session (session 12), all parents showed an increasing trend in their average fidelity score within generalization sessions during the treatment phase. At the 1-month follow up, all parents received average fidelity ratings that were higher than their baseline ratings and four of the eight parents continued to meet fidelity for the intervention.

A multilevel model predicting parents’ average fidelity ratings as a function of treatment phase showed significant differences as a function of phase, F(2, 102) = 25.78, p < .001. Follow-up pairwise comparisons using Tukey’s LSD indicated that the parents’ average fidelity ratings were significantly higher during treatment (M = 3.32, SE = .12) and follow-up (M = 3.60, SE = .33) than baseline (M = 1.87, SE = .18), p < .01; differences between treatment and follow-up were not significantly different. Effect sizes for the difference between baseline and treatment (d = 15.24) and baseline and follow-up (d = 18.18) were large.

Child Spontaneous Language

During baseline, Children 1, 2, 3, 5, and 6 demonstrated low to moderate levels of spontaneous language. Child 4 demonstrated higher levels of spontaneous language, whereas Child 7 and Child 8 used little to no spontaneous language. With the onset of treatment, Children 1, 2, 3, and 5 showed increases in spontaneous language that continued throughout the training program. All four children demonstrated generalization to the home and their spontaneous language rates remained at or above treatment levels at the 1-month follow up. Child 6 also demonstrated increases in spontaneous language within the first few sessions; however, his spontaneous language dropped slightly during the middle of treatment and then increased again when his mother was taught specific strategies for increasing expressive language. His use of spontaneous language declined towards the end of treatment and he did not show generalization of spontaneous language use to the home; however, he demonstrated an increased rate of spontaneous language at follow up. Child 4’s use of spontaneous language remained at or below baseline levels for the first half of the training program. Once his mother was taught specific strategies for increasing his expressive language, his spontaneous language increased almost immediately. Increases in his spontaneous language were observed for the duration of the program, the last two generalization sessions in the home, and the 1-month follow up. Child 7 and 8 showed no changes in spontaneous language in response to the intervention.

A multilevel model with fixed effects indicated a significant effect of treatment on the children’s rate of spontaneous language, F(2, 104) = 8.56, p < .001. Follow-up pairwise comparisons using Tukey’s LSD indicated that the children’s rate per minute of spontaneous language was significantly higher during treatment (M = 1.00, SE = .25, d = .48) and follow-up (M = 1.66, SE = .33, d = 1.44) than baseline (M = .67, SE = .26), p < .01). The children’s rate of spontaneous language was also higher at follow-up than during treatment, p < .01. Effect sizes for the difference between baseline and treatment (d = .48), baseline and follow-up (d = 1.44), and treatment and follow-up (d = .96) were medium to large.

Relationship Between Parent Fidelity and Child Spontaneous Language

We next used multilevel modeling to examine the association between the children’s spontaneous language and their parents’ fidelity ratings. There was a significant relationship between the parents’ average fidelity ratings and their child’s spontaneous language use, b = .11, t(125) = 2.73, p < .01. A model that included all four fidelity dimensions as predictors indicated that both Makes Play Interactive, b = .12, t(136) = 2.71, p < .01, and Helps Increase Complexity of Initiations, b = .13, t(122) = 3.09, p < .01 explained unique variance in child spontaneous language when entered simultaneously (see Table 3).

Table 3 Multilevel modeling regression coefficients and tests of predictors of child spontaneous language use
Full size table

Goal Achievement

Setting goals with parents at the beginning of the training program allowed us to track the children’s progress with respect to initial language goals. Across the eight dyads, 17 initial language goals were set. By the end of the treatment phase, 12 of the initial 17 (71 %) goals had been achieved, with achievement of parent-selected goals seen for all but two children.

Discussion

All parents improved their use of the intervention techniques over the course of treatment. Parents were rarely rated as meeting fidelity for an individual dimension before the techniques comprising that dimension were introduced. After the introduction of each set of techniques, parent use of the corresponding fidelity dimension improved; by week 8 (Teaching Expressive Language), most parents met fidelity for the full intervention. After the introduction of additional topics (Teaching Social Imitation and Teaching Play), parent fidelity decreased slightly. However, fidelity remained consistently above baseline levels. At the 1-month follow-up, four of eight parents met fidelity for the full intervention; the others had average fidelity ratings below the standard fidelity score, but similar to their post-treatment rates and above their baseline rates, indicating relatively strong maintenance of parent skill over 1 month.

Parents in the once per week group required substantially fewer sessions to achieve fidelity than parents in the twice per week group. This difference appeared to be a function of the rate at which new techniques were introduced. The direct teaching procedure was not introduced until week 7 for both groups; this was the point in the program when parents both groups tended to achieve fidelity of implementation. Thus, our data suggest that the parents learned to use the techniques with fidelity as they were introduced and that providing coaching once rather than twice per week does not compromise parent learning.

Visual inspection of the session data indicates that most children demonstrated an increasing trend in spontaneous use of language targets concurrent with the onset of treatment. Reliable gains in spontaneous language could be observed for five of the eight children, with an additional child (Child 4) showing an increasing trend in this behavior after his mother received training in prompting language. Six of the children achieved parent-selected language goals during the study, providing further evidence for clinically significant change in language for the majority of children.

The two children who did not show an improvement in spontaneous language during treatment, Children 7 and 8, were both functionally nonverbal at intake. Child 8 was capable of verbal/vocal imitation at intake and demonstrated improvements in prompted language (not reported here) with the onset of treatment; thus, with continued intervention, it is likely that he would have developed some spontaneous verbal communication. In contrast, Child 7 did not make contingent utterances at intake and did not show an improvement in prompted language during treatment. This might suggest that for children without contingent utterances, training the parent to teach alternative approaches to communication (i.e., sign language, PECS) may be more successful for increasing spontaneous communication.

Although there was evidence of increased rates of spontaneous language targets during treatment for most of the children, the session data were somewhat variable. This is likely an effect of the phased approach in which the intervention was taught, such that parents were asked to practice different strategies each session. Some strategies were likely to facilitate child language use within the session (i.e., prompting language), while others were less likely to do so (i.e., prompting play skills). Additionally, there were changes from session to session in parents’ fidelity of implementation as they learned new techniques. Although overall fidelity improved over the course of treatment, a number of parents demonstrated a “dip” in fidelity as new techniques were taught. This was particularly pronounced towards the end of treatment when strategies for teaching imitation and play were introduced. Finally, parents were only using the full intervention at the very end of treatment; thus, their children were only exposed to the intervention at full intensity for a short period of time. It might be expected that once parents were able to implement the full intervention, improvements in child language would become more pronounced. Indeed, the finding that all of the verbal children demonstrated rates of spontaneous language at the 1-month follow-up that were similar to or higher than their rates at the end of treatment supports this possibility.

A more direct test of the relationship between the parents’ use of the intervention techniques and the children’s rate of spontaneous language targets within session comes from multilevel modeling analyses. There was a significant positive linear relationship between average parent fidelity ratings and child spontaneous language, providing support for the functional relationship between the parents’ use of the ImPACT intervention and improvements in their children’s spontaneous language skills. When the individual dimensions were entered simultaneously into the model, both Makes Play Interactive and Helps Increase the Complexity of Initiations made a unique contribution to predicting child spontaneous language. This finding indicates that both responsive parent behaviors and direct prompting uniquely contribute to the development of spontaneous language, and are consistent with the proposition that combined developmental and naturalistic behavior interventions may be particularly helpful for teaching social communication to young children with ASD (e.g., Ingersoll 2010; Stahmer et al. 2011).

These findings also suggest that some intervention techniques may not be as important for building expressive language skills. For example, modeling and expanding language, a set of techniques that is often used in developmental language interventions (e.g., Hanen), was not found to predict spontaneous language use. Thus, previous associations between this set of techniques and language improvements in children with ASD may due to an association with other more “active” ingredients, such as general responsiveness. Indeed, language interventions employing only modeling and expanding language have tended to show minimal short-term effects in young children with ASD (Ingersoll et al. 2012).

Taken together, these findings have important implications for the use of this intervention in community settings. First, both the original twice per week and the modified once per week individual parent training models were effective for increasing parent use of the intervention. Thus, this adaptation can likely be made to the program without adversely affecting parent learning. However, additional research is needed to determine whether frequency of parent training impacts child spontaneous language, as substantial differences in initial child language ability across the two formats precluded this analysis. Second, there is evidence that several sets of intervention techniques are related to child use of spontaneous language, with higher rates of parent fidelity for those dimensions associated with greater child use of spontaneous language. Thus, a specific effort should be made to ensure that parents achieve a high degree of fidelity on those dimensions in order to maximize the effect of the program on expressive language. Moreover, any “reinvention” of the program by community providers to better fit their intervention setting should ensure that parents achieve high levels of fidelity for those dimensions.

There are several limitations to this study. The training was conducted primarily in a clinic setting and thus included limited measures of generalization to the home. In addition, we did not measure the extent to which parents used the intervention outside of the study. Thus, it is unclear how much overall exposure to the intervention the children received. Although large for a single-subject design study, the small number of participants with similar backgrounds limits the generalizability of this study to the broad range of children with ASD and their families. Finally, although the intervention was developed in a community setting and is likely to be readily transportable, additional research in community settings is needed to ensure that similar gains can be achieved in “real world” settings.

In summary, this study provides initial support for the efficacy of a 12-week parent-mediated social communication intervention for improving spontaneous language use in verbal young children with ASD. Larger scale studies conducted in community settings that can examine a broader array of parent and child outcomes is needed to fully understand the benefit of such an approach for young children with ASD and their families.