Instructive feedback (IF) is an instructional arrangement that involves the presentation of non-target stimuli (hereafter secondary targets) during trial-based instruction (Holcombe et al., 1993; Reichow & Wolery, 2011; Vladescu & Kodak, 2013). Early evaluations of IF were interested in the potential that exposing learners to secondary targets would result in more rapid acquisition when the same targets were later taught (Holcombe et al., 1993; Wolery et al., 1991). Subsequent research has shown that participants may demonstrate mastery of the secondary targets without additional instruction (Anthony et al., 1996; Nottingham et al., 2017; Reichow & Wolery, 2011; Werts et al., 2003). These findings represent an important means for behavior analysts to arrange for efficient instructional gains by producing a greater amount of learning (i.e., number of targets) in a similar amount of time (Wolery et al., 1991).

A common tactic in IF research is to include unmastered primary and secondary targets. To the authors’ knowledge, only three studies have included the presentation of secondary targets with previously mastered primary targets (Frampton & Shillingsburg, 2020; Laddaga Gavidia et al., 2022; Tullis et al., 2019). In each of these studies, an unknown secondary target was presented following a previously mastered primary target. In practice settings, it may be uncommon for previously mastered targets to be repeatedly presented in this manner. Instead, mastered demands may be interspersed during instruction or assessed in a maintenance session. Although it is important to determine the extent to which performance has maintained, maintenance sessions may be viewed as lost instructional time as they represent a period of programming spent without targeting new skills. As such, it would be meaningful to incorporate IF targets during maintenance sessions so that these periods might also result in instructional gains.

When arranging IF during maintenance sessions, two variables may be particularly relevant: the assignment of secondary targets and primary targets (Werts et al., 2003, 2011) and the schedule by which secondary targets are presented (Griffen et al., 1998; Nottingham et al., 2020). The findings of previous research suggests that participants still acquire the majority of secondary targets when presented following any, not a specific, primary target (Werts et al., 2003, 2011). Additionally, Griffen et al. (1998) found that participants demonstrated mastery of secondary targets that were presented intermittently (i.e., following an average of four primary targets) and continuously (i.e., after every primary target) at similar rates. These studies may support the inclusion of secondary targets during maintenance sessions for individuals with disabilities to bolster instructional gains. The purpose of the current study was to evaluate the acquisition of novel secondary targets when presented during review sessions of previously mastered targets. Novel secondary targets were presented during the consequence portion of maintenance trials following an average of three mastered demands. We also extended prior research on IF by including two comparisons with the same participant separated by 24 months.

Method

Participant and Setting

Jane was a White female diagnosed with ASD from an English-speaking home. Jane began Evaluation 1 at four years of age, and she began Evaluation 2 at six years of age. The Verbal Behavior Milestones Assessment and Placement Program (VB-MAPP; Sundberg, 2014) was completed with Jane one month after Evaluation 1 and updated two months before Evaluation 2. During the first evaluation, Jane’s performance on the VB-MAPP was consistent with that of an emerging level two learner. She exhibited strengths in the mand and matching-to-sample domains, receiving all points through level two. She also emitted approximately 25 intraverbal fill-ins. During Evaluation 2, Jane received approximately half of all points in level three on the VB-MAPP, although she only received a single point in the intraverbal domain in this level. Secondary targets were intraverbal fill-ins and intraverbal associations in Evaluations 1 and 2, respectively. Intraverbal targets were selected from Luckevich (2008).

All sessions took place in an individualized setting that included a child-sized table, chair, and shelves for preferred tangible items. Additional chairs were available for independent data collectors.

Dependent Variables and Response Measurement

The dependent variable was unprompted correct responses which were scored if Jane emitted the target response specific to the antecedent verbal stimulus within 5 s. If Jane did not respond within 5 s or emitted any response other than the target response, an unprompted incorrect response was recorded. Jane’s echoic behavior during secondary target presentation was also recorded. Data collectors coded whether Jane echoed the antecedent verbal stimulus (e.g., repeated “thick and...”) or the target response (e.g., “thin”). If she echoed the entire secondary target (e.g., “thick and thin”), both were scored. A mastery criterion was set at one probe session (Evaluation 1) or two consecutive probe sessions (Evaluation 2) with at least 88% unprompted correct responses.

Experimental Design

A multiple-probe design (Horner & Baer, 1978) and multiple-baseline design across target sets was used in Evaluations 1 and 2, respectively. Probes were conducted each day before any maintenance sessions (described below).

Preference Assessment

We conducted a brief Multiple Stimulus Without Replacement preference assessment with Jane before Evaluation 1 (Carr et al., 2000). The assessment was not repeated before Evaluation 2 as Jane would consistently mand for preferred items during reinforcement intervals.

General Procedure

All sessions were conducted during tabletop instruction. Mastered demands included simple listener discriminations (e.g., body part identification) or motor imitation targets. Jane had previous experience with three-step least-to-most prompting hierarchies and prompt-delay procedures but had not previously been exposed to IF before Evaluation 1. Jane received access to preferred items for 20 s after completing an average of three mastered demands (Evaluation 1) or access to preferred toys for two minutes after receiving nine tokens (Evaluation 2).

Instructive Feedback

During IF sessions, mastered targets were presented rapidly and correct responses produced brief praise (e.g., “good”) before the presentation of the next trial. Following an average of three mastered demands, the experimenter delivered praise and 20-s access to a preferred tangible item (Evaluation 1) or a token (Evaluation 2). The secondary target (e.g., “You raise your hand”) was presented concurrently with the reinforcer and the experimenter allowed 3 s for an echoic response. Secondary targets were not assigned to a specific mastered target. Each target set included three secondary IF targets which were presented three times each during a single maintenance session. As a result, maintenance sessions included approximately 27 mastered demands. Maintenance sessions were conducted one to two times each day during appointments that occurred one to three times each week.

Daily Probes

Probes were conducted before training each day. The antecedent verbal stimulus was presented (e.g., “You raise your…”) and Jane was given 5 s to respond. No differential consequences were presented following responses during probe trials. Praise and tangible access or tokens followed correct responses to mastered demands, which were interspersed following an average of three probe trials. During Evaluation 1, probes were conducted separately for each set (i.e., Sets 1–4) and included three presentations of each secondary target in the set for a total of nine trials. Probes for the training set were conducted daily and probes of sets in the staggered panels of the multiple-probe design were conducted intermittently to detect possible threats to internal validity. During Evaluation 2, probes included targets from all sets. Each target was presented in a single trial for a total of nine trials.

Intraverbal Training

Intraverbal training was introduced if performance remained at low levels, or a decreasing trend was observed during IF. Targets were no longer presented during maintenance trials and instead were taught during separate instructional sessions. A constant prompt-delay procedure was used beginning with two sessions presented at a 0-s prompt delay. In these sessions, the experimenter presented the antecedent verbal stimulus (e.g., “you clap your…”) and immediately presented an echoic prompt. Prompted correct responses produced praise and 20 s of a preferred tangible (Evaluation 1) or a token (Evaluation 2). All remaining sessions were conducted at a 5-s prompt delay. If Jane emitted an unprompted correct response, the experimenter delivered praise and 20-s access to a preferred tangible (Evaluation 1) or a token (Evaluation 2). Following an incorrect response or no response, the experimenter re-presented the antecedent verbal stimulus and immediately presented an echoic prompt. If Jane correctly echoed the prompt, the experimenter delivered brief praise and presented the next target. If Jane incorrectly echoed or did not respond following the echoic prompt, the experimenter re-presented the trial at a 0-s prompt delay until a prompted correct response was emitted. Each target was presented three times for a total of nine trials.

Intraverbal Training with Multiple Response Repetition

In Evaluation 2, intraverbal training with multiple response repetition (MRR) was introduced for Sets 2 and 3 because responding remained at zero levels and Jane was leaving for a month-long break. This condition was similar to intraverbal training; however, a more intrusive error-correction procedure was introduced (identified as the most effective procedure for Jane in another assessment). Following an error, the antecedent verbal stimulus and immediate echoic prompt were re-presented three times and Jane was required to echo the target response on all three presentations. Following three prompted correct responses, the antecedent verbal stimulus was re-presented and Jane was given 5 s to emit an unprompted correct response. This procedure was repeated until an unprompted correct response was emitted on the final re-presentation, which produced a token.

Interobserver Agreement and Procedural Integrity

A second independent observer was present during 96.3% and 93.5% of probe and IF sessions during Evaluations 1 and 2, respectively. Trial-by-trial interobserver agreement (IOA) was calculated by taking the total number of trials with an agreement divided by the total number of trials (i.e., nine) and multiplied by 100. Mean IOA was 97.8% (range, 77.8% to 100%) in Evaluation 1 and 98.1% (range, 66.7% to 100%) in Evaluation 2.

An independent observer recorded procedural integrity during 100% and 98.1% of sessions in Evaluations 1 and 2, respectively. This observer recorded whether the experimenter accurately implemented all protocol components on each trial. Procedural integrity was 99.5% (range, 88.9% to 100%) in Evaluation 1 and 99.6% (range, 77.8% to 100%) in Evaluation 2. An additional observer recorded procedural integrity for 97.8% and 91.6% of sessions for Evaluations 1 and 2, respectively, and mean procedural integrity IOA (calculated as described above) was 99.9% (range, 88.9% to 100%) for both evaluations.

Results

During Evaluation 1 (Fig. 1), Jane’s performance during daily probes remained at near-zero levels during the initial baseline phase for all sets. The IF condition produced responding at the mastery criterion for Sets 2 and 4 in six and three daily probes, respectively. Low levels of responding were observed during IF for Sets 1 and 3. Intraverbal training was introduced for both sets and responding met the mastery criterion in six probes for Set 1. After five sessions of intraverbal training for Set 3 targets, the intervention was terminated as pervasive errors to a single target were observed (i.e., Jane said “Bike” following You ride in a… due to previous exposure to You ride a…; this error was addressed outside of this study).

Fig. 1
figure 1

Unprompted correct responses across daily probes for Evaluation 1. Note. *Training discontinued due to pervasive response bias; IV TX = Intraverbal training

Full size image

Figure 2 represents correct responding during daily probes for Evaluation 2. During baseline, no correct responses were emitted. Set 1 targets were exposed to IF, and Jane consistently emitted a single target response on every probe trial. Intraverbal training was introduced, and the mastery criterion was met in five probes. For Set 2, no correct responses were emitted for four consecutive probes. Because Jane was leaving for a month-long break, we introduced intraverbal training for Set 3 targets, while continuing to present the Set 2 targets during IF. Doing so allowed for greater exposure to IF for Set 2 targets and for a preliminary comparison to direct training of targets in Set 3. Nevertheless, after five additional daily probes with no correct responses in either set, intraverbal training and MRR was introduced for both sets. Following training, Jane emitted 100% unprompted correct responses during the post probe.

Fig. 2
figure 2

Unprompted correct responses across daily probes for Evaluation 2. Note. IV TX + MRR = Intraverbal training with multiple response repetition

Full size image

Jane’s echoic behavior following secondary target presentations is shown in Fig. 3. During Evaluation 1, Jane consistently echoed the target response more often than the antecedent verbal stimulus, which occurred during a greater proportion of trials over time. During the second evaluation, Jane echoed fewer than 5% of secondary targets across both sets.

Fig. 3
figure 3

Percent echoic responses during instructive feedback

Full size image

Discussion

The current study serves as an initial demonstration of methods to incorporate IF targets during maintenance sessions. Across two evaluations separated by 24 months, Jane acquired two sets of targets when presented during IF. Three target sets required direct training using a constant prompt-delay procedure and two additional sets required a more intrusive error-correction procedure to produce responding at mastery levels. The current findings are well-aligned with prior research on IF suggesting that learners may not always master the secondary targets (Laddaga Gavidia et al., 2022; Nottingham et al., 2017; Vladescu & Kodak, 2013). In one example, Laddaga Gavidia et al. (2022) observed low levels of responding to secondary targets for one participant in two of three comparisons. For this participant, responding to IF targets at mastery levels in the first set was observed after five probe sessions; however, low levels of responding in the remaining two sets continued despite numerous IF sessions and 10 or more probe sessions. In the current study, the number of probe sessions varied across sets; yet, responding never met the mastery criterion when more than five probes were conducted. The current findings and those of Laddaga Gavidia et al. (2022) may suggest that delayed acquisition of IF targets may not be expected. Instead, if performance does not increase early during exposure to IF, introducing a more intrusive intervention may be warranted or even necessary.

Previous research on IF has inconsistently reported the occurrence of echoic or imitative behavior during secondary-target presentation. When these data are provided, the relation between echoic behavior and acquisition of the secondary targets is unclear. For example, Vladescu and Kodak (2013) and Nottingham et al. (2020) observed high levels of echoic behavior across all participants and also observed acquisition of the secondary targets in nearly every comparison. In contrast, Haq et al. (2017) observed echoic behavior in greater than 80% of trials for one participant, although the secondary targets were never acquired. Laddaga Gavidia et al. (2022) observed similarly high levels of echoic behavior for one participant who did not consistently master the secondary targets. The current study extended prior research on echoic behavior during IF in two ways. First, we separately recorded whether the participant echoed the antecedent verbal stimulus or the target response. Previous research has recorded whether the participant echoed any part of the IF target (Laddaga Gavidia et al., 2022) or has not specified whether a specific portion of the vocal model had to be emitted to qualify as an echoic (Haq et al., 2017; Nottingham et al., 2020; Vladescu & Kodak, 2013). It is also unclear whether a learner echoing the entire secondary target (e.g., “thick and thin”) would result in correct responding during subsequent probes requiring only the target response (e.g., “thin”). During the first evaluation, Jane always echoed the target response more frequently than the antecedent verbal stimulus; however, all instances of echoic behavior during the second evaluation included both the antecedent verbal stimulus and target response. Future research should consider further refinements to measurement systems to allow for a more thorough analysis of the learner’s behavior during secondary target presentations. This may include recording which part of the secondary target is emitted as an echoic, but also other responses that may influence the acquisition of the secondary targets (e.g., attending; Haq et al., 2017).

The current study further extends the literature on echoic behavior during IF due to the unique finding of an increase in echoic responding across sets in the first evaluation. This might illustrate the development of the echoic as a mediating or precurrent response (Parsons et al., 1981; Skinner, 1968; Sundberg et al., 2018). Specifically, the echoic might have emerged because it increased the probability of reinforcement during subsequent probes. Interestingly, the echoic was largely absent during Evaluation 2. It is possible that Jane was emitting the echoic covertly, although the lack of secondary target acquisition may not support this hypothesis. Nevertheless, the emergence of the echoic across training sets in Evaluation 1 serves as a potential demonstration of the development of non-targeted precurrent or mediating behavior during verbal behavior programming.

Limitations of the current study should be mentioned. First, the inclusion of a single participant limits conclusions that may be drawn regarding the generality of these findings. The generality or social validity of our procedures is also unknown. The current study presented IF targets during maintenance sessions aligned with procedures commonly used in our lab; however, these methods may not be representative of those used in other clinical or educational programs. Additional research might further consider how IF targets may be embedded in other maintenance assessments or similar programs that do not typically include learning opportunities (e.g., baseline sessions). In doing so, researchers might come to better understand the conditions in which secondary targets are likely to be acquired and the role that primary targets may serve in the efficacy of IF procedures.

The current study serves as an initial demonstration of methods to embed secondary targets during maintenance sessions for a learner with ASD. Jane showed inconsistent acquisition of secondary targets, requiring a more intrusive intervention for five target sets. Although these targets were infrequently acquired, embedding secondary targets during maintenance sessions may represent a low effort means by which additional learning trials can be presented during clinical programs. These and other methods to embed additional learning opportunities during skill acquisition interventions might bolster behavior analytic instruction aimed to produce the greatest gains for individual learners.