In accordance with the transactional model of child development (Sameroff 2009), children and members of their social environment (parents, family members, teachers) engage in a series of exchanges that shape the child’s behavior, skills, and understanding over time, ultimately contributing to the development of complex cognitive, language, and social competencies. Learning is conceptualized as a fundamentally social enterprise, with the quality and the frequency of these interpersonal micro-encounters accumulating, scaffolding, and yielding a striking transformation in human functionality in a span of just a few short years (Rosenthal and Zimmerman 2014; Sameroff and Fiese 2000).

The presence of an autism spectrum disorder (ASD) can fundamentally alter child development by affecting both the quality and frequency of these social learning exchanges early in development (Freeman and Kasari 2013). Inherent to the disorder are vulnerabilities in social motivation (manifesting as decreases in social initiations, responses to social overtures, and overall reciprocity), which can derail an optimal developmental trajectory and contribute to a range of undesirable downstream effects, including impairments in language, communication, and social skills (Chevallier et al. 2012; Jones and Klin 2009). Fortunately, the field’s recognition of the proximal and distal impact of ASD on child development has yielded systematic research efforts into intervention strategies to correct or at least attempt to minimize this early derailment (see French and Kennedy 2018 for a recent review).

Autism Intervention

Researchers found initial success in using basic behavioral learning contingencies rooted in applied behavior analysis (ABA) to systematically shape behavior over time using repeated learning trials and reinforcement (Lovaas 1987). The introduction of next generation Natural Developmental Behavior Interventions (NDBI; Schreibman et al. 2015) further enhanced these procedures through the use of child-centered strategies that took into account the benefits of child motivation, developmental considerations, everyday learning settings, and parental involvement. The outcomes associated with participation in NDBI models are quite favorable, including more frequent early social learning experiences, increased social development, reduced likelihood of maladaptive behavior, and better generalization (Schreibman et al. 2015).

Pivotal Response Treatment

One NDBI model, Pivotal Response Treatment (PRT; Koegel and Koegel 2006), has a substantial literature of empirical research supporting its efficacy (see Verschuur et al. 2014 for a recent review). PRT combines motivational and behavioral principles in an attempt to maximize child attention and responsiveness to learning opportunities.

Two interrelated concerns of the PRT intervention model are that (a) there is significant variation in the quality of treatment delivered even when fidelity of implementation criteria is technically met, and (b) the model does not explicitly emphasize maximizing a child’s social engagement within current fidelity procedures (Vernon et al. 2012). To be clear, strategies to promote social connection, engagement, and reinforcement have frequently been a part of PRT implementation (particularly with highly experienced parents and clinicians), but these elements have never been specified as necessary for meeting fidelity requirements (Bryson et al. 2007; Koegel et al. 1989). Adult treatment providers can actually meet fidelity for implementation of the PRT procedures (that is, they can serve as active intervention partners) while operating as relatively passive social partners. Specifically, they can provide access to desired stimulus items in response to a child’s initiated or prompted verbal request but are not technically required to be playful, engaging, or emotionally stimulating. Serving merely as a gatekeeper for a child’s favorite toys is unlikely to improve social engagement between adult and child and may actually damage the delicate affiliative bond between the two. If parents and clinicians are effectively interfering with children from having free, unrestricted access to their preferred play materials, this arrangement is very likely to create a negative interpersonal association over time. While many intuitive and experienced clinicians and parents have been observed to augment the standard PRT protocol through the inclusion of playful and engaging social elements, these modifications are not an explicit part of the manualized PRT procedures or the corresponding fidelity criteria.

The variation in PRT implementation styles and differing levels of competence among clinicians and parents may partially explain some of the variance in treatment response. Even in the context of next generation NDBIs, there exists a wide range of child outcomes. Some individuals make rapid gains in language, social, and cognitive milestones, while others are classified as minimal or non-responders with few demonstrable developmental improvements (Sherer and Schreibman 2005). Differential response to treatment drive the need for continued development in the area of ASD intervention science.

Intervention Development and Evaluation

It is logical that the evaluation of a modified behavioral intervention requires empirical investigations on both microgenetic and longitudinal levels. On a microgenetic level, the procedures should yield measurable enhancements to the quality and/or frequency of moment-by-moment behaviors and interactions observed in individual therapy sessions. In other words, quantifiable real-time improvements must be observed between the interventionist (parent or professional) and child. Recently, efforts were made to modify and enhance the PRT model to improve the quality of these within session, micro-level exchanges. In an initial investigation, clinician-child exchanges were examined within the context of a traditional PRT implementation and a modified procedure that emphasized social engagement and social activities as the sole forms of reinforcement (Koegel et al. 2009). Even when the reinforcer strength was held constant across conditions, the child demonstrated measurable improvements in social engagement, eye contact, and directed facial expressions in the modified PRT condition that exclusively used social reinforcement.

A subsequent investigation examined the effects of this intervention modification on parent–child dyads (Vernon et al. 2012). Using a multiple baseline design, the use of traditional and modified PRT procedures were compared in a parent-delivered intervention paradigm. After training parents in traditional PRT procedures and coding social behavior data from both members of the dyad across multiple sessions, parents were then introduced to the modified procedures. Use of the modified procedures yielded significant increases in both child social responses (eye contact, verbal initiations, directed positive affect) and corresponding parent social responses (directed positive affect, synchronous engagement). As a follow-up, time-window sequential analysis procedures were then used to identify the presence of predictable, reoccurring parent–child and child-parent transactions that established clear cause-and-effect social behavior contingencies between both family members (Vernon 2014). In other words, the onset of specific parent actions immediately elicited highly desirable child social responses and vice versa. These improvements were also observed in generalization and short-term follow-up probes taken in the weeks after the intervention concluded.

While these data from past investigations are promising, there remain unanswered questions pertaining to potential long-term developmental benefits of the modified PRT strategies. One must demonstrate that these intervention procedures (that yielded within session therapeutic benefits) are also linked to improvements on outcome measures that adequately capture the constellation of targeted developmental domains. Ultimately, implementation of the intervention must also promote a more favorable developmental trajectory.

In order to develop a rigorously designed randomized clinical trial to examine longitudinal developmental outcomes, a meticulously designed pilot investigation is a crucial first step. Pilot studies are designed to assess key feasibility characteristics of a planned research methodology prior to embarking on a larger study (Moor et al. 2011; Thabane et al. 2010; Van Teijlingen and Hundley 2001). Such studies serve as a trial run of procedures to ensure that potential problems are identified and necessary modifications are made in order to maximize the likelihood of a successful follow-up investigation.

The current pilot study evaluated several critical aspects of clinical trial feasibility across participant factor, treatment protocol, and outcome domains. Specifically, the current project examined participant recruitment, retention, treatment acceptability/tolerance, intensity, and fidelity of implementation factors. Analytic procedures were also used to obtain preliminary efficacy data, effect sizes, and outcome variance information. All efforts were undertaken to inform the design of a future large-scale RCT.

Methods

Research Design

A randomized clinical trial design was used as the methodological framework for this pilot investigation. Random assignment with stratification by age was conducted. When a child met all inclusionary criteria, a coin flip determined if they were randomized to the treatment or waitlist groups for 6 months. The next qualifying age-matched child (with an age match defined as being within 3 months of another participant) was then assigned to the opposite group.

Participants

Thirty-one parent–child dyads were recruited over the course of the two-year trial period. Inclusionary criteria consisted of: (a) an age between 1.5 and 4.5 years (18–56 months) at intake, (b) an autism classification based on cut-off scores of the Autism Diagnostic Observation Schedule, Second Edition (ADOS-2; Lord et al. 2000; Luyster et al. 2009), and (c) an ASD diagnosis based on DSM-5 diagnostic criteria (APA 2013) and expert clinical judgement by a licensed clinical psychologist. Children with comorbid medical or psychiatric conditions were excluded from participation. Participating parents were required to (a) attend a two-day intake evaluation, (b) be available and willing to participate in two hours per week of parent education sessions and, (c) be present for the remaining 8 h per week of clinician-implemented early intervention sessions. Families were permitted to continue with any existing preschool and/or community-based early intervention services (applied behavior analysis, speech-language therapy, and occupational therapy) and the mean number of outside service hours per week did not significantly differ between treatment and waitlist groups.

A final project consort diagram is provided in Fig. 1. Of the 31 parent–child dyads recruited, 28 were eligible for the study and 23 ultimately completed this pilot investigation. This project was approved by the research site’s institutional review board (IRB) and informed consent was obtained from each family.

Fig. 1
figure 1

PRISM trial consort diagram

Full size image

Intake Procedures

Screening

Interested families participated in a phone screen in which project procedures were described in detail and screening questions were asked to assess the family’s likelihood of meeting all inclusionary criteria. Families that tentatively met criteria were scheduled for an intake evaluation, whereas families who did not meet the criteria were given referrals for community services.

Intake Evaluation

Participants who passed the screening were then invited to complete a two-day intake evaluation in which standardized diagnostic, developmental, language, vocabulary, and adaptive functioning measures were administered.

Measures

Mullen Scales of Early Learning

The Mullen is an individually administered comprehensive measure of developmental abilities in infants and preschool children (Mullen 1995). The resulting Early Learning Composite (ELC) Standard Score was used as a global composite of developmental functioning with a mean of 100 and standard deviation of 15. Additionally, four scales (Visual Reception [VR], Fine Motor Skills [FM], Receptive Language [RL], and Expressive Language [EL]) were also examined for more specific information on multiple developmental domains. Each scale is represented with t-scores (M = 50, SD = 10).

Autism Diagnostic Observation Schedule—Second Edition (ADOS-2)

The ADOS-2 is a semi-structured, standardized observational assessment of social communication and behavioral symptoms associated with ASD in individuals aged 12 months through adulthood (Lord et al. 2000; Luyster et al. 2009). One of three modules (Toddler Module; Module 1; Module 2) was administered as appropriate. The total Calibrated Severity Score (CSS; Esler et al. 2015; Gotham et al. 2009) was used as a common metric for comparing ASD symptom severity across modules.

Preschool Language Scales, 5th Edition (PLS-5)

The PLS-5 is a developmental language assessment that evaluates both auditory comprehension and expressive communication skills of children from birth to age 7:11 (Zimmerman et al. 2007). The total score on this measure (mean of 100 and standard deviation of 15) served as the primary measure of participant language gains.

Peabody Picture Vocabulary Test, 4th Edition (PPVT-4)

The PPVT-4 is a norm-referenced, individually administered assessment of single-word receptive language (Dunn and Dunn 2007). This assessment was used as the primary measure of receptive vocabulary. PPVT-4 performances are presented as standard scores (M = 100; SD = 15).

Expressive Vocabulary Test, 2nd Edition (EVT-2)

The EVT-2 is a norm-referenced, individually administered assessment of single-word receptive vocabulary skills (Williams 2007). This assessment was used as the primary measure of expressive vocabulary. EVT-2 performances are presented as standard scores (M = 100; SD = 15).

Vineland Adaptive Behavior Scales, 2nd Edition (Vineland-II)

The Vineland-II Parent/Caregiver Rating Form measures a child’s everyday adaptive skills in the home and community based on parent report (Sparrow et al. 2005). The Vineland-II provides information about adaptive performance in four domains: Communication, Daily Living, Socialization, and Motor Skills. These domains are combined to generate a Vineland Adaptive Composite (mean of 100 and standard deviation of 15), a measure of overall adaptive functioning.

Intervention Procedures

Intervention Conceptual Model

This pilot randomized clinical trial focused on evaluating the feasibility, utility, and preliminary efficacy of the modified PRT procedures, which are collectively referred to as the Pivotal Response Intervention for Social Motivation (PRISM) model. The PRISM model uses the foundation of traditional PRT principles (Koegel and Koegel 2006) to create social communication learning opportunities. These strategies include (a) strong emphasis on child selection of motivating stimulus materials, (b) task variation to maximize motivation, (c) provision of clear antecedent prompts/discriminative stimuli to cue a child to respond, (d) a combination of both simple maintenance tasks and more challenging acquisition tasks to balance engagement with learning, (e) reinforcement of child language attempts without requiring perfectly articulated verbal responses, (f) provision of immediate, contingent access to requested stimuli, and (g) use of motivating reinforcing stimuli that is directly/naturally related to the words spoken by the child. PRT is generally implemented in natural, everyday environments (e.g. participant homes, community settings) and routines (e.g. playtime, mealtimes).

Within the context of a traditional PRT intervention paradigm, learning trials generally occurred in the following format: a clinician/parent arranged the social-communicative learning opportunity in the following three-step contingency (a) the adult presented an antecedent cue to respond (e.g., the adult provided a verbal prompt or enticed the child with a reinforcing object), (b) they waited for the child to make a verbal request attempt, and (c) they reinforced the child’s verbal attempt by delivering the motivating stimulus (often a highly preferred toy or object).

The PRISM model is firmly grounded within a PRT framework with modifications to directly prioritize and target child social engagement. It makes explicit several important components that are not included in the original fidelity procedures, including noncontingent exposure, high affect bids, and social reinforcement strategies:

Noncontingent Exposure

In the PRISM treatment model, the clinician or parent initially provided noncontingent access to a potential activity of interest. This procedure consisted of granting a child free exposure to a variety of activities that were likely to be enjoyable to the individual without requiring an initiated or prompted verbal response. While experienced PRT clinicians and parents often provide this introductory “warm-up” period with new activities, the published fidelity procedures do not accommodate use this strategy, which would technically be coded as a failure to stay contingent.

High Affect Bids

After the child demonstrated interest and engagement in activity as evidenced by close physical proximity and sustained attention, the adult then verbally modeled the name of the activity (verbal prompt) using a high-affect bid, defined as pairing a positive directed facial expression (smiles and/or laughter) with a word or phrase delivered in a higher than usual vocal register (i.e. a playful, motherese-like voice). The purpose of the high affect bid was to increase the social salience of the bid in order to foster social engagement with the child and increase the likelihood of a response.

Social Reinforcement Activity

Following a verbal response or initiation from the child, the adult immediately reinforced these language attempts by engaging the child in carefully constructed social reinforcement activity. The primary requirement of this component was that the adult’s actions were required for reinforcement to take place, thereby making their presence a necessary and integral component of each exchange. In other words, access to these social forms of reinforcement would not be possible if the adult was not present. This modification ensured that children continued to build communication skills while simultaneously forging stronger social connections with parents and clinicians over time. These activities were often visual (e.g., animating a preferred inanimate toy), auditory (e.g., singing a favorite song from the radio), tactile (e.g. tickling a child that enjoys physical contact), or proprioceptive (e.g. swinging the child in the air) in nature. The overarching goal was to augment or enhance any comparable solitary activity that the child might already enjoy (Koegel et al. 2009).

To be clear, these types of social activities are currently used by some PRT clinicians, but without explicitly making them required treatment components reflected in both training and fidelity of implementation procedures, these activities will not be consistently implemented, especially by beginning clinicians and parents. The modified components of the PRISM enhanced model are provided in Table 1.

Table 1 Summary of traditional PRT and PRISM model similarities and differences. Note: Modifications are in bolded text
Full size table

Development of Social Reinforcement Activities

During the initial intervention sessions, parents and clinicians explored each child’s preferences and interests. Parent interviews, naturalistic observations of the child’s play, and introductions of different toys, objects, and activities were implemented. The specific sensory characteristics of preferred items and activities were then identified. These characteristics were then embedded into interactive activities that replicated the appeal of each child’s existing but historically non-social interests.

For example, if the child derived reinforcement from a musical toy, a possible social reinforcement activity might involve having a parent or clinician mimic those sounds following the request of the child. If the child was observed to enjoy water play, a socially equivalent activity might involve a playful social splashing game. Again, the primary requirement was for the adult to serve as an integral part of the interaction and for the social component to augment or enhance the characteristics of a preferred nonsocial activity. Development of additional social activities continued throughout a family’s participation in the trial.

Intervention Implementation

Participants who were randomly assigned to the treatment condition received 6 months (26 weeks) of the PRISM treatment model. They were allocated 10 h a week of intervention: 8 h of one-on-one clinician-implemented treatment and 2 h of parent education in the intervention strategies with the child present. Sessions were delivered in home and community settings and were scheduled in a manner that fit within each family’s weekly routines.

Clinician Training

Lead (parent educator) clinicians were experienced graduate student researchers who (a) were extensively trained and supervised by a licensed clinical psychologist, board certified behavior analyst, and senior PRT clinician, and (b) had previously met PRT fidelity of implementation with five separate children with ASD. Clinicians implementing one-on-one treatment were either graduate student researchers or undergraduate research assistants who had (a) completed a two-day training in PRT, (b) observed 10 h of PRT sessions, and (c) demonstrated PRT fidelity of implementation with at least one child with ASD.

Parent Education

Each family identified one parent who would participate in the parent education sessions for the duration of the trial. The weekly parent education hours were designed to equip caregivers with key therapeutic strategies that they could implement outside of the direct intervention hours, thus increasing the overall intensity of treatment. Within the parent education sessions, the trial clinicians adhered to an established curriculum. After a general introduction to the PRISM concepts in the first session, parent educators discussed the concepts in greater detail during subsequent sessions. They explained the rational for each component, modeled the techniques with the child, encouraged the parents to practice the techniques, and provided in vivo feedback. After one month of parent education sessions, the parent educator gradually faded their level of direct modeling of intervention strategies and encouraged parents to take the lead as primary intervention agents during the sessions. Parent educators discussed use of the PRISM strategies in family activities, aided in troubleshooting any difficulties or barriers, continued to provide ongoing feedback, and jointly developed new activities with the assigned parents.

Fidelity

Fidelity procedures consisted of behavioral coding of a 5-min video recording of parents implementing the PRISM procedures with their child. Each learning trial (defined as a distinct antecedent, behavior, consequence transaction between parent and child) was coded for the presence or absence of necessary components. In addition to the traditional PRT fidelity components of child choice/task variation, child attention, clear opportunity, maintenance/acquisition task, natural reinforcement, immediate/contingent reinforcement, and reinforcement of attempts (see Bryson et al. 2007 and Koegel et al. 1989 for detailed descriptions), the PRISM components of noncontingent exposure, high affect bids, and social reinforcement (defined previously in the Intervention Procedure section) were also coded. Adults were required to demonstrate all of these components in 80% of trials to meet fidelity of implementation, with the exception of maintenance/acquisition and noncontingent exposure components.

For the maintenance/acquisition component, adults were required to alter the complexity of their prompts (i.e. not exclusively use simple maintenance or overly complex acquisition prompts the entire probe). This component was scored on a global pass/fail basis for the entire five-minute probe. For the noncontingent component, adults were required to allow a child to engage freely in each new activity prior to providing any prompts at least 80% of newly selected child-preferred activities.

Post-intervention Procedures

At the conclusion of treatment, all clinical measures were re-administered to assess developmental changes. As previously noted, participants in the waitlist group were then provided the opportunity to receive six months of the enhanced PRISM procedures.

Pilot Study Analytic Procedures

The procedures of this pilot RCT were evaluated using criteria set forth by multiple researchers (Moore et al. 2011; Thabane et al. 2010; Van Teijlingen and Hundley 2001). Feasibility objectives with analytic plans and target criteria were established for three interrelated areas: Participant Factors, Treatment Protocol, and Outcome Domains.

Participant Factors

The Participant Factors domain examined issues related to project throughput and group equity. Within this larger domain, the Recruitment subdomain examined the capacity to recruit and enroll an adequate number of families to fulfill project requirements. A sample size of 12 per group was selected based on existing guidance for pilot studies with advantages for blocking and precision related to standard error (Julious 2005; van Belle 2002). The Randomization subdomain examined the sufficiency of randomization and stratification procedures to yield equitable groups.

Treatment Protocol

The second focus of the pilot study analytic process was the Treatment Protocol, which examined the sufficiency and participant tolerance of the intervention procedures. The subdomain of Acceptability examined data related to the tolerance of the intervention. One component of acceptability consisted of program completion and withdrawal statistics. The other component consisted of post-participation survey responses. After completion of the project, parents were asked to rate how much they agreed with the following statements on a 0–10 Likert scale (0: strongly disagree; 5 neither agree nor disagree; 10 = strongly agree):

  1. 1.

    My family had a positive experience participating in the intervention project

  2. 2.

    The project taught me effective strategies for working with my child

  3. 3.

    My child’s social engagement improved after participation in this project

  4. 4.

    My child’s language development improved after participation in this project

Families were also asked to provide written feedback about their participation. The subdomain of Intensity examined total hours utilized versus hours offered as another dimension of acceptability as well as a metric for fit within family routines. The subdomain of Fidelity examined if parents mastered use of the intervention components at the end of the trial.

Outcome

The final evaluative domain of this pilot study focused on Outcome. Although this pilot investigation was not intended to serve as an efficacy trial, preliminary outcome data were obtained to examine potential impact on developmental measures to inform the implementation in a future large-scale RCT. Because of the sample size of this pilot and previously unknown effect size parameters, mixed Group × Time analytical procedures were not conducted. Instead, the subdomain of Pre–post Analyzes examined baseline to project completion changes within treatment and waitlist groups separately using paired sample T-tests. The Effect Size subdomain was used to understand magnitude of change parameters on the utilized measures. Finally, the subdomain of Variance focused on obtaining information on the range of anticipated range of outcomes associated with treatment exposure by examining the 95% confidence intervals of the resulting effect sizes and examining overlap in these intervals between treatment and waitlist groups.

Results

Participant Factors

Recruitment

Recruitment strategies initially consisted of email announcements, direct mailings, print advertisements, and direct communications with state regional centers, early childhood educators, and pediatricians. Recruitment efforts were later expanded to prioritize targeted social media posts. This revised recruitment approach was found to be very effective, ultimately facilitating the recruitment of 31 dyads during the two-year trial period. In order to compensate for the disproportionate number of dropouts in the treatment group, following a dropout, newly recruited children with the same age match were paired with an existing waitlist child. This procedure ultimately resulted in 16 participants being assigned to the treatment group and 12 assigned to the waitlist group.

Randomization

The baseline participant characteristics are summarized in Table 2. There were no significant between-group differences on the demographic variables of age, sex, or racial category. While there were no significant between-group differences on the ADOS-2, Mullen, PLS-5, PPVT-4, or EVT-2 (p > .05), there was a significant difference in pre-trial Vineland-II ABC Standard Scores, with Treatment Group participants having higher scores than Waitlist Group participants, M = 12.56, 95% CI [5.15, 19.98], t(21) = 3.52, p = .002.

Table 2 Pre-trial between group demographic and measure comparisons
Full size table

When examining measure subscales/subdomains, there were no significant between-group differences on the Mullen and PLS-5 subscales (p > .5). There were significant between-group differences on multiple Vineland-II subdomains, with the Treatment Group participants having higher scores than Waitlist Group participants on the Vineland-II Communication (M = 14.02, 95% CI [4.30, 23.75], t(21) = 3.00, p = .007), Daily Living (M = 10.46, 95% CI [1.79, 19.12], t(21) = 2.51, p = .02), and Socialization (M = 12.71, 95% CI [5.94, 19.49], t(21) = 3.90, p = .001) subdomains.

Treatment Protocol

Acceptability

As depicted in the consort diagram, 16 family dyads were assigned to the treatment condition. Two families (12.5%) discontinued the study prior to the start of the intervention. One family was unresponsive to the research team’s communication attempts after intake. The other family lived in a geographically distant location and indicated that after further consideration, the 2–3 h daily commute to which they had originally agreed made participation unfeasible. Two additional families (12.5%) began the treatment phase but withdrew before the trial was completed. One family specified family factors as reason for discontinuing (i.e. a divorce resulting in joint physical custody across two distant cities). The other family discontinued after their child began to experience chronic seizures and required acute medical care. A total of 12 families (75%) completed the treatment protocol.

The 12 families in the treatment group provided the following agreement ratings on a 0–10 scale (0 = strongly disagree; 5 = neither agree nor disagree; 10 = strongly agree): Family had a positive experience, M = 9.83 (SD = 0.41); Learned effective strategies, M = 9.00 (SD = 1.10); Social Engagement Improved, M = 9.17 (SD = 1.17); Language Development Improved, M = 9.00 (SD = 1.26).

Dosage

The 12 families that completed treatment completed a mean of 177.70 h (SD of 43.97) hours of the possible 260 h, or a mean of 68.35% of total allocated hours. This equates to a mean of 6.81 of the 10 h/week. Four families (25%) met the benchmark of 80% of allocated treatment hours and 8 families (75%) fell under this threshold.

Fidelity

The parents of the 12 families that completed treatment demonstrated use of a mean of 85.13% (SD of 12.07%) of the treatment procedures during their final two parent fidelity probes. Ten of the 12 families (83.33%) fell above the established PRT fidelity threshold of 80%.

Outcome

Pre–post Analyses

Results of the pre to post-intervention analyses are summarized in Table 3. In the treatment group, statistically significant changes from baseline were found for the ADOS-2 CSS, Mullen ELC, PLS-5 Total Score, and PPVT-4 Standard Score. Significant changes were not found on EVT-2 Standard Score and Vineland-II Adaptive Behavior Composite scores. No significant changes from baseline were observed on any measures in the waitlist group.

Table 3 Pre and post trial assessment measure data for treatment and waitlist groups
Full size table

In secondary analyses of measure scales/subdomains, significant changes from baseline were observed in the treatment group on the Mullen scales of VR, FM, and RL, along with the Vineland-II subdomain of Communication. In the waitlist group, significant pre-post changes were observed in the Mullen scale of FM skills.

Effect Size

Cohen’s d was calculated for each pre-post measure change and is displayed in Table 3. The treatment group experienced changes associated with medium to large effects across all measures that had significant pre-post changes (ADOS CSS d = − 1.41; Mullen ELC d = 0.72; PLS-5 d = 0.57; PPVT-4 d = 0.59). In the secondary measure scales/subdomains, the treatment group experienced medium to large effects across all scales with significant pre-post changes (Mullen VR d = 0.56; Mullen FM d = 1.08; Mullen RL d = 0.76; Vineland-II Communication d = 0.75).

Variance

The 95% confidence intervals for effect size are also summarized in Table 3. When examining the lower bound of the confidence intervals for measures with significant pre-post changes, four of the six measures fell above the threshold for a small effect (d > 0.20): ADOS-2 CSS, Mullen ELC, PLS-5, and PPVT-4. In the subscales, four scales/subdomains with significant pre-post changes fell above the small effect threshold for the lower bound: Mullen VR, Mullen FM, Mullen RL, and Vineland-II Communication scales/subdomains.

When conducting between group comparisons, there was no overlap in the effect size 95% confidence intervals between treatment and waitlist groups on two measures: the ADOS-2 CSS and Mullen ELC. There was a marginal overlap of 0.01 on two additional measures: the PLS-5 and the PPVT-4. On the scales/subdomains, there was no confidence interval overlap between treatment and waitlist groups on the Mullen VR and FM scales.

Discussion

The objective of this investigation was to systematically examine procedures and outcomes of a pilot randomized clinical trial for feasibility, utility, and preliminary efficacy across the domains of participant factors, treatment protocol, and outcome.

The data suggest that adequate recruitment is possible for the intended population of interest using the current inclusionary criteria. Specifically, target recruitment numbers were met within the specified project timeline. Current randomization and age stratification procedures yielded groups that were comparable on the majority of the primary measures. However, there were significant pre-trial differences on the Vineland-II and a trend toward some additional pre-trial between group differences. These observed differences may be an artifact of the relatively small sample size used in this pilot investigation, which increased the likelihood of spurious differences despite the use of randomization procedures. However, as an additional safeguard against between-group pre-trial differences, the use of developmental standardized scores will be considered for future stratified random assignment plans. Specifically, Mullen, PLS-5, and Vineland performances will all be considered as possible stratification factors.

As a component of treatment acceptability, the total completion rate for families assigned to the treatment group was 75%, which fell below the specified goal of 80% trial completion. It is encouraging that the stated reasons for not beginning or withdrawing participation in the trial were primarily attributable to logistical (e.g. drive time) and personal/medical factors (e.g. divorce, seizure onset), rather than treatment acceptability factors. However, one family did not specify a reason for not proceeding with treatment after assignment, and it is also possible that other families were not always forthcoming with concerns they had about the treatment protocol.

The high level of parent-reported treatment acceptability and efficacy on the anonymous post-trial surveys were also encouraging. Families who completed the trial made very favorable endorsements, which suggest that the social validity of the procedures is quite high. In addition to positive ratings, families frequently left very positive written feedback. One parent wrote, “The program was perfect. It was extremely useful to my son and our family.” Another parent stated, “This project was a vital piece in my son’s development. We watched our son flourish in ways we could have only hoped. The level of care and understanding we experienced was amazing and truly life changing.” Another comment read, “I had a very positive experience and enjoyed the people that worked with our family.” However, because post-project parent ratings and comments were not obtained from families who did not start or who discontinued the intervention, it is possible that the resulting endorsements were biased, as they only reflected the perspectives of families who successfully completed the trial.

In a related area of treatment dosage, it appears that most families did not reach the project’s target intensity of 80% of total allocated hours. Since few families surpassed the dosage target, it appears that the project’s specified treatment intensity may not be congruent with the logistics of participating family routines and schedules. Preschool schedules, other therapies, sibling extracurricular activities, family vacations, and parent/child illnesses were the most commonly cited reasons for missed or reduced sessions. However, given the preliminary outcome data, the implemented intervention model may yield promising effects even with the unplanned reduction in treatment intensity. Less intensive exposure to the PRISM protocol may still yield promising developmental benefits, especially when parents are equipped with the skills to continue treatment delivery between scheduled clinical sessions. A reduction in weekly hours on future projects may be justified based on the data.

Fidelity data supports the claim that most participating families met criteria with independent delivery of the treatment procedures following their participation in the six-month trial. Parents demonstrated mastery of the components needed to deliver the PRISM procedures accurately and consistently. However, variation in the mastery of specific principles still warrants additional modifications and monitoring to the training procedures. Additionally, future research should examine the progress of parent fidelity throughout the six-months of treatment. It was observed that some parents master fidelity much earlier than others, which may warrant modification of parent education sessions, including earlier fading of modeling and introduction of increasingly complex treatment concepts.

Pre-to-post project analyses provide preliminary evidence of efficacy across several of the utilized measures. The enhanced PRT package may yield reduced ASD symptomology and improvements to core developmental domains. The effect size confidence intervals across measures were also very encouraging. While researchers are strongly discouraged from using effect size values obtained from pilot studies to estimate population parameters (Thabane et al. 2010), the confidence intervals of these effects provide a reasonable estimate of the potential range of outcomes for future large-scale investigations. The obtained effect sizes and corresponding confidence intervals suggest that PRISM holds promise in enhancing developmental outcomes and suggests that further evaluation of this intervention model in an expanded trial may be a worthwhile pursuit.

A summary table of the pilot study analytic plan, results, and proceed/modification decisions that resulted from these analyses are provided in Appendix A.

Limitations and Future Directions

While the results of this pilot study are promising, it has several important limitations. It is worth reiterating that due to the focus on feasibility rather than efficacy, the study design focused on primarily on procedural rather than outcome variables. Because of power concerns, definitive claims of treatment efficacy cannot be made. Subsequent implementation of a sufficiently powered RCT will require a multi-pronged recruitment strategy consisting of (a) formal partnerships with large volume recruitment sources (e.g. local hospitals and medical practices), (b) multiple research sites across diverse geographical locations, and (c) an expansion of the total project duration.

There is also an unanswered question related to the unique additive role of the modifications in the PRISM protocol compared to traditional PRT. Because the traditional PRT and PRISM procedures were not directly compared in this RCT, we cannot say with certainty that comparable results would not have been obtained by simply using the traditional PRT approach. A direct comparison was not pursued in this trial, as it was anticipated that directly comparing the two approaches for evidence of superior developmental and behavioral gains would require a sample size and time period not feasible within the scope of this pilot study. As previously described, micro-analytic studies contrasting traditional PRT versus PRISM approaches have yielded evidence of superior within-session parent and child responses. It is reasonable to assume that these observed within-session improvements to engagement (i.e. increases in eye contact, direct facial expressions, verbal initiations, and reciprocity) would ultimately yield more optimal developmental trajectories, but this hypothesis could not be tested by the current investigation.

Implications

The outcomes from this investigation were highly informative and largely support the feasibility of a follow-up large-scale trial. The preliminary findings suggest that these strategies may hold promise for altering the developmental trajectories of young children with ASD. Data suggestive of reductions in ASD symptom severity and gains in key developmental and adaptive domains may emerge after only six months of intervention. If a more rigorous trial confirms these preliminary findings, the enhanced PRT procedures may offer a strategy for further improving developmental outcomes of children with ASD.

Due to the scaffolding nature of development, there is a growing understanding that the initial symptoms of ASD can inhibit and delay the establishment of subsequent capacities, creating a detrimental effect on downstream development (Jones and Klin 2009; Muratori and Maestro 2007). The clarified and enhanced PRISM procedures have previously been shown to improve the quality of clinician-child and parent–child transactions at a micro-exchange, within-session level. It is conceptualized that such exchanges may serve to reestablish and amplify social motivation within children with ASD, which is suspected to be both a pivotal area of child development (Koegel et al. 2001) and a well-established etiological theory of autism (Chevallier et al. 2012). Targeting social motivation in children with ASD should arguably be the main objective of early intervention efforts, as interpersonal engagement appears to be the primary catalyst for acquiring developmental competencies. This trial served as an initial step in examining whether increased attraction to and participation in moment-by-moment social exchanges can accumulate and facilitate the acquisition of a more favorable developmental trajectory over time.