There has been substantial growth in the prevalence of autism (Maenner et al., 2021), with high levels of unmet service needs for autistic children and their families (Karpur et al., 2019), particularly among families in rural and medically underserved communities (Drahota et al., 2020). Thus, systematic research focused on developing and improving strategies for dissemination and implementation of evidence-based autism services is a high priority.

Parent-mediated interventionFootnote 1 (PMI) is one cost-effective and ecologically valid way to increase access to evidence-based autism intervention. Research suggests that parents can learn to use evidence-based intervention strategies with a high degree of fidelity, with their children experiencing concomitant gains in language and social communication development (Brian et al., 2017; Pickles et al., 2016), adaptive skills (Scahill et al., 2016), and decreases in disruptive behavior (Bearss et al., 2015). PMI can also improve parent well-being by increasing parent self-efficacy and reducing parenting stress (Iadarola et al., 2018; Ingersoll et al., 2016). Despite these benefits, widespread dissemination of PMI is challenging, due to a shortage of trained professionals, lengthy waitlists, and family resource limitations (e.g., high service costs, limited transportation to services, lack of childcare during appointments) These limitations are even more pronounced for families who live in rural and underserved areas (Straiton et al., 2021; Suppo and Floyd, 2012).

Telehealth has the potential reduce barriers and increase access to evidence-based services. Recent systematic reviews have demonstrated that telehealth-based PMI is highly acceptable to parents and can result in improvements in parent implementation of evidence-based intervention techniques, parent self-efficacy and stress, and child social communication skills and disruptive behavior (Ellison et al., 2021; Sutherland et al., 2018). Nevertheless, most studies are characterized by small samples, lack of control groups, non-randomized designs, and limited follow-up. Indeed, Simacek and colleagues (2021) identified only three studies which employed a randomized controlled trial (RCT), and results were “positive, yet mixed” (p. 20). Community use of telehealth for autistic children has increased dramatically with the onset of COVID-19 and stay-at-home orders (Dueñas & D’Agostino, 2022; Simacek et al., 2021), indicating a need for more rigorous research on its efficacy.

Telehealth interventions can be either self-directed, in which participants engage with the interactive program at their own pace, or therapist-assisted, in which they receive additional guidance and feedback from a trained professional as they complete the program. Self-directed programs have far greater dissemination potential as they do not require a trained professional and can typically be administered at a lower cost. However, research on telehealth CBT interventions for mood and anxiety have found therapist-assisted programs lead to better client outcomes than self-directed programs (Andersson & Cuijpers, 2009; Spek et al., 2007). There has been limited research comparing self-directed and therapist-assisted telehealth PMI for autism; however, research suggests that parent coaching with a trained professional may be necessary for some, but not all parents, to successfully implement evidence-based intervention techniques with fidelity (Wainer & Ingersoll, 2015). A better understanding of the contributions of the different programs, as well as which families will benefit from each, will make it possible to develop more cost-effective delivery models with different levels of support depending on family needs (Phaneuf & McIntyre, 2011; Steever, 2011).

Project ImPACT is a manualized, evidence-based, parent-mediated, Naturalistic-Developmental-Behavioral Intervention (NDBI) that targets social communication skills in young autistic children and children with an increased likelihood of developing autism (Ingersoll & Dvortcsak, 2010, 2019). ImPACT Online is an adapted version of Project ImPACT, with 12 interactive, web-based lessons (Ingersoll and Berger, 2015); it can be administered as a standalone intervention (self-directed program) or in combination with telehealth coaching with a therapist (therapist-assisted program). A pilot RCT with 28 families of autistic children demonstrated feasibility and acceptability of the self-directed and therapist-assisted programs (Ingersoll & Berger, 2016; Pickard et al., 2016), and suggested that both led to improvements in parent learning and well-being, and children’s social communication skills (Ingersoll et al., 2016). However, parents in the therapist-assisted group made greater gains in intervention fidelity and perceptions of the positive impact of their child, and their children had marginally greater gains in language and social skills than parents in the self-directed group. These findings are promising and consistent with two recent non-randomized, comparison group design studies that have demonstrated the benefit of using telehealth to provide coaching in Project ImPACT (Hao et al., 2021; Li et al., 2022).

Here, we present the findings for parent outcomes from a randomized comparative efficacy trial of self-directed and therapist-assisted ImPACT Online. We examined the relative effect of the self-directed and therapist-assisted models on two primary outcomes: parent intervention fidelity and parent self-efficacy, as they are targeted by the intervention and are thought to influence child outcomes. We also examined the programs’ effect on three secondary outcomes: intervention knowledge, positive impact of the child, and parenting stress, all of which were found to improve in our pilot RCT of ImPACT Online (Ingersoll et al., 2016).

Identifying moderators of treatment can help determine who is most likely to benefit from telehealth-based PMI and who may need additional support. Previous research has identified parenting stress as a moderator of autism intervention outcomes for parents (Estes et al., 2021) and children (Rickards et al., 2007). We hypothesized that parenting stress may be an important moderator of parent outcomes for ImPACT Online, and that parents who are more stressed at intake may be less likely to benefit from the self-directed than the therapist-assisted model. Thus, we examined whether pre-treatment parenting stress moderated treatment effects on parent outcomes.

It is also important to understand factors that influence parent engagement with telehealth programs, especially given our previous finding that parent program engagement was predictive of improvements in parent intervention knowledge and fidelity (Ingersoll & Berger, 2015). In our pilot trial, we found that parents in the therapist-assisted group engaged with the online lessons more in terms of number of logins, duration on site, and learning activities completed, and were more likely to complete the program than parents in the self-directed group. Thus, we anticipated that we would find a similar influence of therapist assistance in our current study. Similarly, parent expectancies about positive outcomes of an intervention predict engagement in traditional parent-mediated interventions (Nock et al., 2007) and telehealth-based programs (Meischke et al., 2011). Further, treatment acceptability (Barbosa et al., 2012) and program satisfaction (Brinkmeyer et al., 2004) are associated with client engagement in behavioral and medical treatments and were associated with program engagement in our pilot RCT (Ingersoll & Berger, 2015). Parents’ perceptions of barriers during treatment also predict their engagement in parent training interventions (Kazdin et al., 1997). Thus, we examined the extent to which these factors were associated with program engagement for the self-directed and therapist-assisted groups. Although our pilot study did not find an association between program engagement and computer/internet fluency (Ingersoll & Berger, 2015), we re-examined this association in the current study, since research suggests that computer self-efficacy is associated with the acceptance of telehealth (Or & Karsh, 2009). Finally, we examined whether any sociodemographic variables predicted program engagement as an exploratory aim.

Method

Design and Procedure

This study used a randomized controlled trial with intent-to-treat analysis. Participants (child with a primary caregiver) were randomly assigned to receive: (1) self-directed ImPACT Online (SD), (2) therapist-assisted ImPACT Online (TA), or (3) information and resource support (IRS; control). The trial protocol was registered at ClinicalTrials.gov (NCT02721381) and approved by the Institutional Review Board at Michigan State University. Enrollment ended early due to the COVID-19 pandemic; thus we were not able to recruit our targeted sample size of 90 families. Participants were assessed at intake (T1), after 6 months of intervention (T2), and after a 3-month follow-up (T3). Assessments were administered by trained examiners masked to group assignment. After the T1 assessment in the lab, children were stratified by age (18–35 months; >36 months) and developmental quotient (DQ < 55; DQ ≥ 55) and then randomly assigned to one of the three groups using random permuted blocks of 6. See Fig. 1 for participant flow through the study. All children continued receiving their existing educational and intervention services. All families who expressed a need were provided with the necessary technology for the duration of their participation: laptop (n = 8); webcam (n = 1); high-speed internet service plan (n = 2).

Participants

Children between the ages of 18 and 96 months and their primary caregiver were recruited from community agencies and professionals serving young autistic children in Michigan. Recruitment focused on underserved communities as defined by residence in a Rural Health and/or Health Provider Shortage Area by the U.S. Health Resource & Services Administration; however, families did not have to live in an underserved community to be enrolled. One family who lived outside of Michigan participated after hearing about the study on Clinicaltrials.gov. To be eligible, children were required to have a community diagnosis of Autism Spectrum Disorder (ASD), not yet exhibit conversational speech, and meet criteria for ASD on Autism Diagnostic Observation Schedule-2 (Lord, Rutter et al., 2012). Children with a history of significant brain injury, known neurological or genetic condition, significant sensory or motor impairment, or major medical problems were excluded as they might be expected to respond differently or require program modifications. Parents had to be proficient in English, although other languages could be spoken in the home. Seventy-four children were assessed for eligibility; an additional four families did not complete the eligibility assessment due to a no show (n = 1) or discontinuation due to the initiation of the stay-at-home order due to COVID-19 (n = 3). Nine children did not meet inclusion criteria (n = 8) or declined to participate (n = 1). After randomization, one family declined to participate and did not complete T1 assessments. The final sample included 64 families.

Primary caregivers (53 mothers, 9 fathers, 1 grandfather, and 1 foster mother) ranged in age from 24 to 54 (Mean = 35.27 years). Parents were predominantly female (84%) and non-Hispanic white (75%). The racial composition of parents was 80% white, 9% Asian, 6% Black/African American, 2% American Indian/Alaskan Native, and 3% more than one race and the ethnic composition was 6% Hispanic/Latino, and 94% Not Hispanic/Latino. Parent racial/ethnic demographics were similar to the demographics in Michigan, with the exception of a higher rate of parents identifying as Asian and lower rate identifying as Black/African American. Although all parents had to speak English to participate, 20% percent also spoke a second language in the home. 52% of parents had a 4-college degree or higher, 72% were married and living with a partner, and 59% were employed full- or part-time. Consistent with our recruitment goal, 81% of parents lived in a rural or underserved area.

Children ranged in age from 18 to 93 months (M = 47.09 months). Children were predominantly male (77%) and non-Hispanic white (66%). The racial composition of children was 69% white, 8% Asian, 8% Black/African American, and 16% more than one race, and the ethnic composition was 11% Hispanic/Latino and 89% not Hispanic/Latino. Child racial/ethnic demographics were similar to the demographics in Michigan, with the exception of a higher rate of children identified as Asian and More than one race and a lower rate of children identified as Black/African American. Our sample also had a higher rate of children identified as Hispanic/Latino. The children’s mean nonverbal mental age was 26.47 months and verbal mental age was 21.44 months. In terms of language development, 31% were preverbal, 25% were at the one-word stage, 28% were at the word combination stage, 8% were at the simple sentences stage, and 5% were at the complex sentences stage (i.e., Tager-Flusberg et al., 2009). The average number of intervention hours per week that the children received outside of the study was 16.10 (range < 1 to 47 h). Participant groups did not significantly differ on any of the demographic and pre-treatment outcome variables. Participants who did not complete data collection (n = 7) did not differ from participants who completed data collection on any demographic or pre-treatment outcome variables.

Measures

Eligibility and Sample Characteristics

Children were administered the ADOS-2 and the Mullen Scales of Early Learning (n = 63; Mullen, 1995) or Wechsler Preschool & Primary Scale of Intelligence, 4th Edition (n = 1; Wechsler, 2012) in the lab to determine study eligibility and characterize their developmental functioning at T1. Once per month, parents provided information on hours per week of all non-study treatments that their children received throughout study participation (e.g., speech-language therapy, therapeutic preschool). These data were averaged to provide an overall estimate of hours of outside treatment per week across the study period. See Table 1 for participant demographic information.

Parent Intervention Fidelity

The Project ImPACT Intervention Fidelity Checklist (Ingersoll & Dvortcsak, 2019) was used to evaluate the parents’ use of the intervention strategies with their child at T1, T2, and T3. Parents were videorecorded during a parent-child interaction in the family home, in which they were asked to (1) play with their child for 10 min; and (2) have a small snack or meal with their child for 10 min. Parents were instructed to interact with their child as they typically would at home during both activities. Trained undergraduate research assistants blind to group assignment and time point scored each of the five Project ImPACT F.A.C.T.S. strategies on a scale of 1 (“does not implement the strategy, or almost all attempts to use the strategy are incorrect.”) to 5 (“implements the strategy effectively throughout the session”). Scores for each strategy were then averaged to form an overall fidelity rating for each activity. Reliability was calculated for 25% of the observations across time point and conditions using intra-class correlation (ICC = 0.87). Ratings for the play and snack routines were averaged to form an overall fidelity rating for each time point. In cases in which one of the observations was missing, a single observation was used.

Parent Self-Efficacy

Parents completed the Parenting Sense of Competence Scale (PSOC; Gibaud-Wallston and Wandersman, 1978). The PSOC measures the extent to which parents believe they have the skills and knowledge needed to be good parents (efficacy) and their perceptions regarding the value of parenthood (satisfaction). Parents rated 17 items from 1 (“Strongly agree”) to 6 (“Strongly disagree”). Items are summed, with higher scores indicative of higher parenting self-efficacy. Cronbach’s alphas ranged from 0.82 to 0.83 across time points.

Intervention Knowledge

At T1 and T2, parents completed the ImPACT Knowledge Quiz (Ingersoll & Berger, 2015), a 20-item, multiple-choice test that measures comprehension of the Project ImPACT intervention strategies.

Positive Impact

Parents completed the 8-item Positive Impact subscale of the Family Impact Questionnaire (FIQ; Donenberg and Baker, 1993) as a measure of their perception of their child’s positive impact on their experience as a parent. Parents endorsed items comparing their experience with their child to parents with same-aged neurotypical peers on a 4-point scale; higher sum scores indicate greater positive impact. Cronbach’s alphas ranged from 0.85 to 0.88 for the positive impact subscale.

Parenting Stress

Parents completed the Parenting Stress Index, Short Form (PSI-SF; Abidin, 1995), a 36-item measure of parenting stress. Parents rated items on a 5-point scale. Items are summed with higher scores indicating greater parenting stress. The Total Stress scale was used as a measure of parenting stress. Cronbach’s alphas for the PSI-SF ranged from 0.87 to 0.90 across time points.

Computer/Internet Fluency

Parents completed a modified version of the Computer-Email-Web Fluency Scale (CEW; Bunz, 2004; Ingersoll and Berger, 2015) at T1 to assess their level of comfort using the computer and internet. Responses to 13 items were summed, with higher scores indicating greater fluency with technology. Cronbach’s alpha for this scale was 0.63.

Treatment Expectancy

Parents completed the Credibility and Expectancies Questionnaire-Parent Version (CEP-Q; Nock et al., 2007) immediately after random assignment. Parents rated 6 items on a 9-point scale to assess the extent to which they expected treatment would improve their child’s behavior. Items were summed, with higher scores indicating more positive expectations about the treatment. Cronbach’s alpha for the CEQ-P was 0.86.

Treatment Acceptability and Program Satisfaction

Parents in the SD and TA groups rated the acceptability of the intervention using an early version of the Scale of Treatment Perceptions (STP; Berger et al., 2016) at T2. Parents rated 21 items that assessed the effectiveness, family fit, and safety of the intervention on a 7-point scale, with higher scores indicating greater treatment acceptability. Cronbach’s alpha for this scale was 0.72. The average of two additional items: (1) I used the intervention with my child regularly; and (2) I would recommend this program to other parents of young children with social-communication difficulties, was used as a measure of program satisfaction (Ingersoll & Berger, 2015).

Barriers to Treatment

Parents in SD and TA groups rated a modified version of the Barriers to Treatment Participation Scale (BTPS; Kazdin et al., 1997). The BTPS is a well-validated measure of common barriers to participation in child treatment. Items were modified to reflect potential barriers associated with the use of a telehealth-based PMI (as opposed to clinician-led, clinic-based therapy), resulting in the removal of 17 items assessing therapist- and clinic-related barriers, and the addition of 5 items assessing technology-related barriers (Ingersoll et al., 2017). Parents rated 32 items on a scale of 1 (“Never a problem”) to 5 (“Very often a problem”), with higher scores indicating a greater number of perceived barriers. Cronbach’s alpha for this scale was 0.90.

Program Engagement and Intervention Completion

Program engagement was assessed via ImPACT Online’s electronic tracking of user behavior, which identified the percent of learning activities accessed across the 12 lessons. Criteria for intervention completion differed by group. Parents in the SD group were considered to have completed intervention if they accessed at least 75% of the program content and attended at least 75% of the monthly support calls. Parents in the TA group were considered to have completed intervention if they accessed at least 75% of the program content and attended at least 75% of the coaching visits (Ingersoll & Berger, 2015). Parents in the IRS group were considered to have completed intervention if they logged into the resources page at least once and attended at least 75% of monthly support calls.

Group Assignment

Self-Directed ImPACT Online (SD)

Parents in the self-directed group (n = 24) received access to the ImPACT Online program for 6 months. The web application contained 12 self-directed lessons, each of which took approximately 75 min to complete, as well as a Video Library and a Resources page with information about autism (see Ingersoll and Berger, 2015 for a detailed description). The lessons covered the benefits of parent-mediated intervention, how to set social communication goals, how to set up the home for success, as well as the F.A.C.T.S. intervention strategies:1) Focus on your child; 2) Adjust your communication; 3) Create opportunities; 4) Teach new skills; and 5) Shape the interaction. Parents were encouraged to complete one lesson per week and to practice the intervention techniques with their child between lessons. Parents were able to contact project staff via phone or email for assistance with technology-related problems and they received a monthly support call from staff (see IRS description below). However, they received no additional support in learning the intervention.

Therapist-Assisted ImPACT Online (TA)

Parents in the therapist-assisted group (n = 22) were given access to the ImPACT Online program for 6 months and were encouraged to follow the same pace as the self-directed group. Parents also received 2, 30-minute remote coaching sessions per week (24 total sessions) via Zoom by a trained masters-level clinician to assist them in learning the intervention (see Ingersoll and Berger, 2015 for a detailed description). Therapists rated their fidelity to the coaching procedure at the end of each session. Our prior work demonstrated a very high correspondence (97.8%) between self-rated coaching fidelity and ratings of coaching fidelity by independent observers (Ingersoll et al., 2016). Average self-assessed fidelity across sessions was 99.20% (range 95–100%).

Informational Resources and Support Group (IRS)

Parents in the IRS group (n = 18) were given access only to the Resources page of the ImPACT Online program (see above). In addition, parents received a monthly phone call from a masters-level clinician who could answer questions and provide internet resources for autism-related services. This group was used to control for participant maturation as well as the potentially confounding effect of having access to autism-related information and contact with research staff.

Analytic Strategy

IBM SPSS 28.0.0.0 was used to conduct all analyses. The data were inspected for outliers (> 3.5 standard deviation units from the sample mean for a given variable) and deviation from normality (skewness > 1, kurtosis > 3). There was a single outlier for one of the variables (STP), which was removed prior to conducting analyses. To verify that randomization produced equivalent groups, one-way ANOVAs and chi square analyses were conducted on T1 data as appropriate. The primary analyses were conducted with the full intent-to-treat sample (Moher et al., 2001). We utilized multilevel modeling (MLM), which accounts for repeated measurement within participants and allows for the inclusion of participants with missing data.

A series of MLM analyses using restricted maximum likelihood estimation were used to examine the effect of group assignment on our primary and secondary parent outcomes. Time point (T1, T2, and T3) was coded as a categorical variable. Change over time across groups was estimated with the main effect of time, and between-group differences in change in outcomes were estimated with the time × group interaction. Estimates of effect size (Cohen’s d) were calculated using the approach recommended by Feingold (2009), which accounts for change in outcomes as an effect of time and group assignment. Only medium to large effects were interpreted. The primary analyses were also conducted with the intent-to-treat sample using the last observation carried forward method to replace missing values, as well as a completer analysis (including only those individuals who completed assessment at all three time points regardless of degree of program completion). The results of both approaches were highly similar to the MLM results; thus, only the MLM results are presented.

Next, we considered whether pre-treatment parenting stress moderated the effects of time and the time × group interaction on the primary and secondary parent outcomes. Pre-treatment parenting stress was grand-mean centered and included in the MLM models as a covariate. The pre-treatment parenting stress × time interaction was used to determine whether pre-treatment parenting stress moderated change over time, irrespective of treatment group. The three-way interaction of pre-treatment parenting stress, group, and time was used to examine whether the relationship between group assignment and change over time (i.e., group slopes) varied by pre-treatment parenting stress. Significant interactions were followed up with simple slopes (using ± 1 standard deviation) to clarify the nature of the interaction.

Descriptive statistics were used to examine program engagement, completion, treatment acceptability, program satisfaction, and barriers to treatment. Independent t-tests and chi square tests were used to compare groups on these measures as appropriate. To examine predictors of program engagement for the parents in the two ImPACT Online groups, initial bivariate correlations were conducted with program engagement and our predictors of interest (therapist assistance, computer/internet fluency, treatment expectancy, treatment acceptability, program satisfaction, and barriers to treatment), as well as parent and child demographic variables. A hierarchical linear regression was then used to examine predictors of parents’ program engagement. All predictor variables were mean-centered. Demographic variables that were significantly associated with program engagement were entered in the first step and our predictors of interest were entered in the second step.

Results

Effect of Group Assignment on Primary Outcomes

The effect of group assignment on our primary parent outcomes was examined using MLM, with a significant time × group interaction indicative of an effect of group assignment. For parent intervention fidelity, there was a significant effect of time (F(2, 113) = 46.30, p = < 0.001) and a time × group interaction (F(4, 113) = 2.74, p = .03). Parent intervention fidelity scores were significantly higher for the TA group than the IRS or SD group at T2 and T3, with large effect sizes (ds = 0.84 to 1.58). The SD and IRS groups were not significantly different from each other. These results were highly similar when we examined the play and snack observations separately. For parent self-efficacy, there was not a significant effect of time (F(2,108) = 1.14, p = .32) or time × group interaction (F(4,108) = 0.34, p = .85). See Tables 2 and 3.

Effect of Group Assignment on Secondary Outcomes

The effect of group assignment on our secondary parent outcomes was examined using MLM, with a significant time × group interaction indicative of an effect of group assignment. For intervention knowledge, there was a significant main effect of time (F(1,53) = 20.94, p < .001) and a time × group interaction (F(2,53) = 5.34, p = .008). Intervention knowledge scores for the TA group were significantly higher than for the IRS group at T2, with a large effect (d = 1.77). Scores for the SD group at T2 fell between the scores for the IRS and TA groups, but were not significantly different from either; however, there was a medium-to-large effect (d = 0.77) favoring the SD over IRS group.

For positive impact, there was no main effect of time (F(2,104) = 0.53, p = .59), however, there was a significant time × group interaction (F(4, 104) = 2.64, p = .038). Follow up tests did not reveal significant group differences at any time points. However, an inspection of effect sizes revealed a medium effect favoring the TA (d = 0.67 T2; d = 0.74 T3) and SD groups (d = 0.60 T2; 0.59 T3) compared to the IRS group.

For parenting stress, there was no significant main effect of time (F(2,106) = 2.23, p = .11) or time × group interaction (F(4,106) = 0.93, p = .45). However, an inspection of the mean values suggests that findings were trending in the expected direction, with parents in the TA and SD groups showing decreased stress over time, while the IRS group remained high. This observation was supported with a medium effect size favoring the TA group over the IRS group at T3 (d = 0.60).

In our pilot study, program completion predicted gains in intervention knowledge and parent fidelity after controlling for group assignment (Ingersoll & Berger, 2015), highlighting the importance of exposure to the intervention content for increasing parent learning. To examine this possibility more closely, we re-ran our analyses with only those participants who completed their assigned intervention condition (N = 49). The main findings regarding time and time × group interactions were replicated in this subsample. In addition, we found that, while the TA group continued to show greater gains in intervention fidelity compared to the IRS group at T2 (TA: M = 4.30, SE = 0.32; IRS: M = 3.38, SE = 0.18, p < .001) and T3 (TA: M = 4.23, SE = 0.33; IRS: M = 3.23, SE = 0.17, p < .001), it was no longer significantly different from the SD group at either time point (T2: M = 3.80, SE = 0.36, p = .19; T3: M = 4.02, SE = 0.37, p = 1.00). Further, the SD group now exhibited significantly greater gains in intervention fidelity than the IRS group at T3 (p = .019). For intervention knowledge, the TA group continued to show significantly greater gains at T2 (M = 16.35, SE = 1.08) than the IRS group (M = 12.55, SE = 0.53, p = .004), but not the SD group (M = 16.00, SE = 1.29), p = 1.00. However, the SD group now demonstrated significantly greater gains than the IRS group, p = .026.

Moderation of Parent Outcomes by Pre-Treatment Parenting Stress

MLM was used to examine whether pre-treatment parenting stress moderated the effects of time and the time × group interaction on the parent outcomes. Pre-treatment parenting stress did not moderate the rate of change in outcomes by group assignment for any of the parent outcomes (all Fs < 2.09, all ps > 0.05). However, it did moderate the main effect of time on parent self-efficacy (F(2,101) = 6.94, p = .002), positive impact of child (F(2,97) = 7.75, p < .001), and parenting stress (F(2,105) = 8.28, p < .001). Follow up analyses indicated that parents who began treatment with high parenting stress (1 SD above the mean) had greater increases in parent self-efficacy over time than parents who began treatment with low parenting stress (1 SD below the mean) regardless of group assignment. Similarly, parents with high parenting stress at T1 had a greater increases in their ratings of positive impact and greater reductions in parenting stress over time than parents with low parenting stress.

ImPACT Online Program Engagement and Intervention Completion

Descriptive statistics were used to examine program engagement and intervention completion for the two ImPACT Online groups. Independent t-tests and chi square tests were used to compare the two groups on these measures as appropriate. Across the two treatment groups, parents accessed an average of 76% (SD = 32.93) of the ImPACT Online learning activities. Parents in the TA group accessed significantly more of the program (M = 96.29, SD = 7.29) compared to the SD group (M = 58.57, SD = 36.93), (t(44)=-4.70, p < .001). 50% of parents in the SD group completed intervention, compared to 95% of parents in the TA group. A chi-square test of independence indicated that parents in the SD group were significantly less likely to complete treatment than the parents in the TA group, χ2 (11.70, N = 46), p = .001. 89% of the IRS group accessed the Resource page and completed at least 75% of the phone calls.

Treatment Acceptability, Program Satisfaction, and Barriers to Treatment for ImPACT Online

Descriptive statistics were used to examine treatment acceptability, program satisfaction, and barriers to treatment for the two ImPACT Online groups. Independent t-tests were used to compare the two groups on these measures. The overall ratings of the acceptability of the ImPACT intervention (M = 4.80, SD = 0.41) and program satisfaction (M = 6.07, SD = 1.07) were positive, and few barriers to treatment were reported (M = 1.65, SD = 0.50). The TA group (M = 4.92, SD = 0.52) and SD group (M = 4.76, SD = 0.41) did not differ in their ratings of treatment acceptability, t(38)=-1.04, p = .15. However, the TA group was more satisfied with the ImPACT Online program (M = 6.42, SD = 0.88) than the SD group (M = 5.74, SD = 1.16), t(36)=-2.04, p = .048. The TA group (M = 1.41, SD = 0.34) also reported fewer barriers to treatment than the SD group (M = 1.88, SD = 0.53), t(36) = 3.22, p = .003.

Predictors of Parent Program Engagement

Predictors of parent engagement for the two ImPACT Online groups were examined using initial bivariate correlations betweeen parent engagement, parent and child demographic variables, and our predictors of interest, followed by a hierarchical linear regression. Of the parent and child demographic variables, only parent age was associated with program engagement, r(46) = 0.39, p = .008. Therapist assistance (r(46) = 0.58, p < .001), treatment expectancy (r(40) = 0.36, p = .024), treatment acceptability (r(40) = 0.32, p = .046), and program satisfaction (r(38) = 0.58, p < .001) were all significantly associated with program engagement; however, barriers to treatment (r(38)=-0.29, p = .074) and computer/internet fluency were not (r(46) = 0.16, p = .30). A hierarchical linear regression with parent age in the first step and therapist assistance, treatment expectancy, treatment acceptability, and program satisfaction in the second step explained 58% of the variance in the outcome. Parent age (β = 0.32, t = 2.80, p = .009), therapist assistance (β = 0.37, t = 3.03, p = .005), and program satisfaction (β = 0.42, t = 2.90, p = .007) were each unique predictors of program engagement. See Table 4.

Discussion

This study sought to replicate and extend our previous findings comparing the self-directed and therapist-assisted models of the ImPACT Online program on parent learning and well-being. In the present study, we included a control group (i.e., information and resource support group), follow-up assessment, and a larger sample size, although recruitment ended early due to the COVID-19 pandemic.

Parent Learning

We examined one primary measure (intervention fidelity) and one secondary measure (intervention knowledge) of parent learning. Parents in the therapist-assisted group saw greater improvement over time in intervention knowledge and fidelity than parents in the control group; this improvement was maintained through follow up. This finding, along with other recent findings of positive outcomes of using telehealth to provide coaching in Project ImPACT and related interventions (Li et al., 2022; Hao et al., 2021; Simacek et al., 2021), provide strong evidence that parent-mediated NDBIs can be effectively delivered via telehealth-based coaching. When considering the whole sample, we also replicated our previous finding that parents in the therapist-assisted group demonstrated similar gains in intervention knowledge but greater improvements in intervention fidelity compared to parents in the self-directed group. These results suggest that therapist assistance is especially useful for increasing parents’ fidelity to an NDBI.

Contrary to our hypothesis, when considering the whole sample, we did not find a significant difference in rate of change in parent fidelity between the self-directed and control groups. Similarly, gains in intervention knowledge for the SD group were not significantly different from the control group. These findings suggest that our previous conclusion of a benefit of the self-directed program on parent learning may have been premature. At the same time, while not statistically significant, there was a medium-to-large effect size favoring the self-directed group for intervention knowledge, and results were trending in the expected direction for intervention fidelity, especially by the follow-up assessment. Therefore, our lack of significant findings may have been due to our limited sample size. Furthermore, our completer analyses indicated that the self-directed group did demonstrate greater gains in intervention knowledge and fidelity (by follow up) than the control group, and greater gains in the therapist-assisted group compared to the self-directed group were no longer as pronounced. Thus, our failure to demonstrate a benefit of the self-directed program in our full sample was likely due, at least in part, to low program completion in the self-directed group. Indeed, only 50% of the self-directed group completed the program compared to 95% of the therapist-assisted group. Our completion rate for the self-directed group in the current study (50%) was substantially lower than the pilot study (69%), which was likely influenced by COVID-19. For example, several parents in the self-directed group stopped engaging with the program at the start of stay-at-home orders. Interestingly, the stay-at-home orders did not adversely affect engagement for the families who were in the therapist-assisted group at the time.

Other studies of parent-mediated NDBIs have demonstrated improvements in parent intervention fidelity following self-directed programs (Nefdt et al., 2010; Wainer & Ingersoll, 2015). This leaves open the possibility that parents may still effectively learn to use NDBI techniques from self-directed programs, provided they engage fully with the instructional content. However, it also calls into question whether a self-directed program is feasible for most families, particularly during times of high stress. Thus, future research should include larger samples which use robust research designs and observational measures, and measure program engagement. In addition, future research should evaluate which strategies are most likely to increase parent engagement in self-directed programs.

Parent Well-being

We examined one primary measure (parent self-efficacy) and two secondary measures (positive impact and parenting stress) of parent well-being. In our pilot study, we observed similar improvements over time in parent self-efficacy in both the self-directed and therapist-assisted groups. In our current study, we did not replicate this finding; parent self-efficacy remained stable in all groups. This finding is somewhat surprising given the positive effects of general parenting programs on parent self-efficacy for families of children with disabilities (Hohlfeld et al., 2018); however, the impact of parent-mediated NDBIs on parent self-efficacy has not always been observed (e.g., Estes et al., 2014), and less is known about the influence of telehealth-based PMI.

We partially replicated our finding regarding improvements in parents’ perceptions of the positive impact of their child on their experience as parents. Parents in the therapist-assisted and self-directed groups both reported more positive impact of their child over time, while the parents in the control group did not. In our pilot study, gains on this measure were only observed for the therapist-assisted group; in the present study, both the self-directed and therapist-assisted group showed a moderate benefit compared to the control group. We did not observe significant group differences in changes in parenting stress over time, which stands in contrast to our pilot results. It is possible that this was due to our use of a different measure of parenting stress in the current study (PSI-SF was used instead of the negative and social impact subscales of the FIQ). However, inspection of the mean values suggests findings were trending in the expected direction, with parents in the therapist-assisted and self-directed groups showing decreased stress over time and parenting stress in the control group remaining high. Further, there was a medium effect size favoring the therapist-assisted group over the control group at follow-up (d = 0.60). Thus, our lack of a significant finding may also be due to a lack of power.

Taken together, our current results suggest a benefit of telehealth-based PMI on parents’ perceptions of positive impact of their child, with a possible benefit on parenting stress as well. Most studies of parent well-being in autism have focused on the negative impact of a child’s disability on parenting (i.e., parenting stress, anxiety, depression). Research suggests that when a child has a disability, positive and negative views of parenting may coexist (Blacher & McIntyre, 2006), and that parents’ positive perceptions may buffer the effect of stress and enhance well-being (Horsley & Oliver, 2015). Further, research suggests that parents of children with disabilities who perceive greater positive impact experience higher family quality of life (Bayat, 2007; Ferrer et al., 2017). Thus, increases in parents’ subjective feelings of positive impact of the child may be a particularly important collateral effect of parent-mediated interventions. Additional research is needed to better understand the effect of parent-mediated NDBI on parents’ positive and negative perceptions of parenting and how these views interact over time in promoting family quality of life.

Contrary to our predictions, pre-treatment parenting stress did not moderate the impact of treatment on any parent outcomes. Previous research found that parents who began a low-intensity program with higher parenting stress experienced greater improvements in parent self-efficacy than parents who began treatment with lower parenting stress (while the opposite was true for parents who began a high intensity intervention; Estes et al., 2021). We expected that we would find similar results in relation to therapist assistance. However, it is possible that since both of our intervention programs were low intensity, pre-treatment parenting stress may not have had as much of an impact on parent outcomes. Pre-treatment parenting stress did predict rate of change on measures of parent well-being, independent of group assignment; parents with higher levels of stress at pre-treatment reported greater gains in self-efficacy and positive impact of their child, and greater decreases in parenting stress over time than parents who reported lower initial levels of parenting stress. Pre-treatment parenting stress was negatively associated with our other measures of well-being. Thus, those parents with higher stress levels (and lower self-efficacy and positive impact) at the beginning of treatment may have had more room to improve or simply experienced a regression to the mean. Either way, our findings suggest that high parenting stress is not associated with poorer parent outcomes for telehealth-based PMI. Additional research is needed to better understand who is most likely to benefit from self-directed and therapist-assisted programs.

Program Engagement

We examined several predictors of program engagement to identify factors that influence parents’ use of telehealth-based PMI. Treatment expectancy, treatment acceptability, and program satisfaction all positively correlated with program engagement, although only program satisfaction was a unique predictor in regression analysis. This finding suggests that developers should consider these factors in the development of telehealth-based programs. Similar to our previous study, therapist assistance was unique predictor of program engagement. The ImPACT Online program was asynchronous, allowing parents to access the intervention content on their own schedule. Indeed, our previous study suggested that parents often completed the program outside of business hours (Ingersoll & Berger, 2015) and parents reported this aspect to be a significant benefit of the program (Pickard et al., 2016). However, our findings suggest that therapist support may be necessary for some parents to complete the program. This is further underscored by our finding that parents who received therapist assistance reported fewer perceptions of barriers to treatment and greater overall program satisfaction. This highlights the importance of therapist support in helping parents work through an asynchronous program.

Of all the sociodemographic variables we examined, only parent age was associated with program engagement. This lack of clear sociodemographic predictors is consistent with our previous findings (Ingersoll & Berger, 2015) as well as research with other health information technologies (Glasgow et al., 2011; Meischke et al., 2011), suggesting that families from a wide range of backgrounds may be similarly likely to engage. Further, the lack of an association between computer/internet fluency and program engagement may suggest that lower fluency may not be a specific barrier, provided families are given access to the relevant technology and given initial support in using it. Additional research that can examine these questions with a more socioeconomically, racially, and ethnically diverse sample will be important for better understanding the scalability of telehealth-based PMI.

Limitations

There are several limitations to this study that should be acknowledged. First, while the parents in the TA group (and those who completed the SD program) demonstrated greater improvement in their use of the intervention compared to the IRS group, their scores still showed room for improvement. For example, at T3, 47% of the parents in the TA group met intervention fidelity (average score > 4 across play and snack observations). This suggests that additional strategies may be needed to help some parents reach fidelity, even when receiving therapist assistance. Of note, fidelity scores post-intervention for the TA group were similar to a trial of Project ImPACT (Stadnick et al., 2015) which used in person coaching, suggesting additional supports may be important for both telehealth and in person coaching. Second, we had missing data, particularly for some of the parent self-report measures. While MLM is fairly robust at handling missing data, our conclusions would be stronger with a more complete dataset. In addition, due to COVID-19, we were unable to recruit our target sample size (n = 90) and were thus underpowered to observe effects in the moderate range. Our effect size analyses provided some additional interpretation; however, additional research with a larger sample size would be helpful. Moreover, although the majority of our sample resided in an underserved area, our participants were not fully representative of the United States in terms of socioeconomic and racial/ethnicity diversity. Finally, in this paper we focused exclusively on parent outcomes. We acknowledge that an examination of child social communication outcomes is also necessary. Our analysis of child outcomes is underway and will be an important step in understanding the efficacy of the self-directed and therapist-assisted programs.

In conclusion, our findings suggest that ImPACT Online resulted in large positive effects on parent learning, particularly with therapist assistance, and moderate positive effects on some aspects of parent well-being. While parents who complete a self-directed program may also demonstrate increases in parent learning, therapist assistance is likely needed for many parents to fully benefit.

Table 1 Participant Demographic Information
Full size table
Table 2 Results of the MLM analyses and post hoc tests for the primary and secondary outcomes
Full size table
Table 3 Effect sizes estimates for group differences over time for primary and secondary outcomesa
Full size table
Table 4 Predictors of Program Engagement
Full size table
Fig. 1
figure 1

Consort Diagram depicting participant flow through the study

Full size image