Abstract
Language delays are commonly displayed by children on the autism spectrum. To help facilitate the development of verbal behavior, practitioners often implement intensive one-on-one, face-to-face instruction. However, the COVID-19 pandemic hindered typical face-to-face service delivery and caused practitioners to assess alternative approaches to facilitate clients’ continued progress. Instructive feedback (IF) is one teaching strategy to enhance instruction or make it more efficient. During this teaching procedure, instructors provide formal teaching of target responses and embed demonstrations of secondary target responses within sequences of instruction. In the current study, we investigated the efficacy of IF provided within telehealth instruction. Four participants on the autism spectrum participated in the study. Participants received two forms of telehealth instruction that targeted speaker-responding. The first form consisted of discrete trial instruction (DTI), and the second form combined DTI with IF. These results indicate that both forms of instruction improved speaker-responding of primary targets for all participants. Additionally, a secondary analysis of secondary targets indicated that two of the four participants acquired some secondary targets. These results suggest that including IF within DTI might be beneficial for some participants receiving DTI via telehealth.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
Introduction
Telehealth is a growing medium for service provision within the behavioral health field (Centers for Medicare and Medicaid Services, 2018). The purpose of telehealth is to provide clients with access to services (e.g., assessment, diagnosis, intervention, and consultation) that would otherwise be delayed (Nickelson, 1998). An increased reliance on telehealth services for early intervention coincided with the start of the COVID-19 pandemic (Ellison et al., 2021). However, telehealth has potential continued utility as a service format as clients and clinicians regularly encounter barriers that disrupt consistent attendance to in-person instructional sessions (e.g., long-distance to the service provider, car trouble, household obligations). Despite the growing use of telehealth services, only a limited number of studies have analyzed the efficacy of behavioral instructional approaches (e.g., discrete trial instruction) via telehealth (Ferguson et al., 2020; Nohelty et al. 2021; Pollard et al., 2021).
Discrete trial instruction (DTI) is an effective approach to early intervention for children on the autism spectrum (Smith, 2001). DTI involves a structured teaching approach in which the instructor provides repeated exposures to instructional trials. During instructional trials, the instructor presents an antecedent stimulus and provides a specific allotment of time for the learner to respond (e.g., a progressive prompt delay that starts at 0 s and increases by 2 s increments, a constant prompt delay of 10 s). If necessary, the instructor may provide a prompt during the response interval to assist the learner in responding appropriately. If the stimulus evokes an appropriate response, the instructor provides the learner with a preferred stimulus (e.g., social praise, tangible stimulus). Alternatively, if the stimulus does not evoke an appropriate response (e.g., incorrect response or no response), no contingent access to preferred stimuli is provided, and the instructor presents an error correction procedure. While DTI is a clinically supported approach for teaching academic and social skills, practitioners may encounter barriers relating to a client’s limited availability when implementing DTI via telehealth (Hyland et al., 2022). For example, a learner may display a decreased rate of acquired skills if extenuating circumstances (e.g., inconsistent family schedules, limited transportation to service providers) lead to insufficient time being allocated toward face-to-face instruction (Love et al., 2009). Therefore, practitioners may consider efficient instructional approaches that can be feasibly provided through telehealth.
Instructive feedback (IF) is a supplemental teaching strategy that has been used during DTI that may promote the efficient acquisition of both primary (i.e., trained) and secondary (i.e., untrained) targets (Frampton & Shillingsburg, 2020; Nottingham et al., 2015). IF procedures embed presentations of secondary targets within instructional trials (Reichow & Wolery, 2011). These presented responses may occur during the antecedent event (i.e., before the learning trial) or during the consequence event (e.g., before reinforcement is delivered). For example, an instructor might teach a learner to identify features of animals (i.e., the primary targets) and use IF procedures to state letter sounds (i.e., the secondary targets) after reinforcement is delivered. The instructor would first present the instruction (e.g., “Name something that a bird has.”). If the learner engages in the correct intraverbal response (e.g., “Birds have wings.”), the instructor provides a preferred consequence (i.e., reinforce the response) and states the secondary target (e.g., “The letter ‘A’ says, ‘/a/.’”). During IF, the learner is not required to engage in an echoic response when instructors present a demonstration of the secondary target response, and the instructor does not provide a reinforcer if the student echoes the secondary target response.
When implementing IF procedures, instructors may select secondary targets that target listener or speaker responding (Loughrey et al., 2014; Tullis et al., 2017). Additionally, instructors might select secondary targets that demonstrate expansion, parallel, or novel targets (Werts et al., 1995). An expansion target is a secondary target response that does not share point-to-point correspondence with the primary target response, but the secondary target is related to the primary target. For example, an instructor may teach a student to tact individuals dressed as various occupations (e.g., doctor, firefighter). During demonstrations of the secondary target response, the instructor indicates a function associated with the displayed occupation (e.g., “A doctor helps you when you are sick.” ). A parallel target is a secondary target response that shares point-to-point correspondence with the primary target response, but the secondary target does not share formal similarity with the primary target. For example, an instructor might teach students to tact the number eight when presented with an Arabic symbol (i.e., 8; primary target) and demonstrate the tact of the number eight when testing the Roman numeral (i.e., VIII; secondary target). Finally, a novel target is a secondary target response that does not share point-to-point correspondence with the primary target response, and the secondary target does not relate to the primary target. A novel target is demonstrated in the previous example, where the instructor taught the student to identify a feature of a bird and then demonstrated a letter sound within the IF procedure. Previous research has demonstrated that secondary targets that share a relation to the primary targets (e.g., expansion target) have resulted in the emergence of responses under intraverbal control (Frampton & Shillingsburg, 2020; Gavidia et al., 2022; Tullis et al., 2021, 2022). In addition, researchers have observed the emergence of secondary targets that do not share a relation to the primary targets (i.e., novel target) when the target response is under tact control (Nottingham et al., 2020) and intraverbal control (Vladescu & Kodak, 2013).
Previous research indicates that when IF procedures are implemented during in-person DTI, learners acquire secondary targets, and little to no difference is observed in the acquisition of the primary targets (Albarran & Sandbank, 2019; Grow et al., 2017). To date, one study has evaluated the effects of IF implemented through telehealth (Ferguson et al., 2020). Ferguson et al. (2020) evaluated the efficacy of IF implemented through a telehealth arrangement for three dyads of children on the autism spectrum (i.e., six total participants). Researchers taught each dyad to tact superhero characters. During the consequence event, researchers presented an intraverbal response that demonstrated expansion targets that corresponded to the target superhero's power (i.e., the secondary target). The results of training suggest that students within each dyad demonstrated the acquisition of both the primary and secondary targets. Despite these promising results, additional research is necessary to further evaluate the utility of IF within one-on-one telehealth sessions. Additionally, researchers may consider evaluating the acquisition of secondary, novel targets.
The purpose of our study was to further evaluate the effects of telehealth-based DTI both with and without IF on the acquisition of primary and secondary targets. Specifically, researchers assessed two outcomes. First, we evaluated the efficacy of telehealth-based DTI alone and DTI with IF on the acquisition of primary targets. Next, we evaluated the effect that IF had on the acquisition of secondary, novel targets when IF is delivered through telehealth.
Methods
Participants
Researchers recruited four 4–5-year-old individuals diagnosed with autism spectrum disorder (ASD) to participate in this study. The participants had previously received face-to-face applied behavior analysis (ABA) services, including instruction via DTI, from a university-based early intervention (EI) program. Additionally, researchers observed that all participants had an established echoic repertoire (i.e., participants echoed complete sentences consisting of at least five words), and caregivers indicated that participants had no prior exposure to formal instruction that used IF. Eligible participants had access to a stable internet connection, and a caregiver was present for each session. Prior to the initial baseline session, each participant and caregiver met with the primary researcher who certified that all participants met the following inclusion criteria: (1) attended (i.e., eye gaze directed to relevant materials) to digital materials for at least 12 consecutive trials and (2) produced a vocal tact of previously mastered targets, presented through a tablet, for at least 10 out of 12 trials.
Board Certified Behavior Analysts (BCBAs) that oversaw eligible participants completed the Verbal Behavior Milestones Assessment and Placement Program (VB-MAPP; Sundberg, 2008) prior to the participant’s acceptance in the study (see Table 1 for VB-MAPP scores associated with each targeted verbal operant). Jose was a 5-year-old white, European American boy enrolled in a public-school kindergarten classroom. Jose previously received two years of one-on-one DTI and natural environment teaching (NET) instruction. Jose displayed an emerging basic conversation repertoire, and before each session, he infrequently engaged in reciprocal conversations. Elmer was a 4-year-old white, European American boy enrolled in a half-day EI program at a university-based preschool. Elmer previously received one year of one-on-one DTI and NET structured instruction. At the time of the study, Elmer attended a university-based preschool for 12 hours a week and received NET structured, small-group instruction. Elmer displayed an established conversation repertoire and frequently engaged in reciprocal conversations. Sabrina was a 5-year-old white, European American girl enrolled in a public-school kindergarten classroom. Sabrina previously received two years of one-on-one DTI and small group NET structured instruction. Sabrina displayed an emerging basic conversation repertoire and infrequently engaged in reciprocal conversations. Stacy was a 5-year-old white, European American girl. Stacy was homeschooled by her mother. Previously, Stacy received two years of small-group NET. Stacy displayed an established conversation repertoire, and before each session, she frequently engaged in reciprocal conversations.
Setting and Materials
The first and third authors acted as instructors throughout the study. Instructors conducted all sessions on the ZoomTM Video Communications, Inc. (Zoom) platform. Instructors accessed Zoom calls from their homes or a private office at a university-based preschool, and participants’ caregivers connected to Zoom calls from their homes. Jose and his caregiver completed their sessions either at their kitchen table or in their home office. Elmer and his caregiver completed their sessions in their dining room, living room, or bedroom. Sabrina and her caregiver completed all their sessions exclusively at their kitchen table, whereas Stacy and her caregiver conducted their sessions in their living room. Two participants connected to Zoom calls through an electronic tablet (e.g., iPad), and two participants connected via home computers. To protect each participant's privacy, instructors used a HIPAA-compliant Zoom account. Additionally, instructors agreed to end research sessions if a third party (i.e., an individual not affiliated with the study) entered the instructional space within the participant or instructor’s setting.
All digital stimuli were organized and presented in trials according to procedures described by Mattson et al. (2020)—a protocol for developing DTI procedures via Google Slides (Google LLC, 2006). The slide decks that instructors presented during probe sessions included a final slide that displayed a variety of preferred videos, and participants selected one to watch following the session. Prepared materials were presented using Zoom’s “Share Screen” feature. During all calls, one caregiver sat with the participant and delivered preferred edible items.
Response Measurement
The primary dependent variable was the percentage of correct independent primary target responses emitted during each training session. The primary target responses for Jose, Elmer, and Stacy were vocal textual responses that corresponded to the presented word, letter, or letter blends (e.g., reading the word), and the primary target responses for Sabrina were vocal intraverbal responses that corresponded to the presented instruction (see Table 2). For each trial, observers scored a correct independent response if the participant produced the target vocal response within 5 s of the initial target stimulus presentation or an incorrect response if the participant did not produce the target vocal response. Following each session, researchers calculated the percentage of correct independent primary target responses by dividing the total target independent responses by the total trials and multiplying the quotient by 100. The mastery criteria for all primary target stimulus sets was two consecutive sessions with correct independent responses above 90% or five consecutive sessions above 80%.
The secondary dependent variable was the percentage of correct secondary responses emitted during each probe session (see Table 2). The secondary target responses for all participants were intraverbal responses that corresponded to novel targets (i.e., primary and secondary targets shared no relation). Similar to primary target responses, observers scored a correct independent response if the participant produced the target vocal response within 5 s of the secondary target. Researchers used probe sessions to evaluate the emergence of all secondary target responses.
Interobserver Agreement (IOA) and Procedural Integrity
Instructors recorded the DTI, DTI with IF, and probe conditions through Zoom. Secondary independent observers collected data on all dependent variables and procedural integrity for at least 30% of the recorded sessions. Researchers randomly selected sessions from a list of saved session recordings. Across all observed sessions, researchers defined an agreement as any trial in which the instructor and independent observer coded matching responses. Researchers calculated trial-by-trial interobserver agreement (IOA) by taking the number of agreements, dividing them by the total number of trials, and multiplying the quotient by 100. Mean IOA across all participants was 99.3% (range, 83.3-100% [see Table 3]).
Independent observers recorded the instructor’s implementation of condition-specific variables (see Table 4). Researchers calculated procedural integrity by dividing the sum of condition-specific variables implemented correctly by the total condition-specific variables per session and multiplying the quotient by 100. Mean procedural integrity across all conditions was 99.7% (range, 91.8–100% [see Table 5]).
Experimental Design
Researchers employed an adapted alternating treatments design (i.e., AATD [Sindelar et al., 1985]) embedded in a non-concurrent multiple baseline design across participants. Researchers utilized a non-concurrent multiple baseline design to compare the effects each teaching procedure (i.e., DTI or DTI with IF) had on the acquisition of their respective primary target responses. Additionally, researchers employed the adapted alternating treatments design to compare the effects of the teaching procedures on the acquisition of the secondary targets.
Stimulus Sets
Researchers emailed each participant’s caregivers to identify appropriate targeted responses for primary and secondary targets. Researchers requested that caregivers identify potential targets that had not been directly taught to the participant and that might meet the participant’s current academic/behavioral goals (i.e., assessed skill deficit needing to be trained). Potential targets required participants to engage in either textual or intraverbal responding. Once researchers identified a list of potential targets, researchers requested that the caregivers not teach the selected targets outside of the study, and instructors probed the potential targets and assessed each participant’s responding. Researchers listed all potential stimuli that did not evoke a correct response and paired potential stimuli according to their topographical features and corresponding response topography (Cariveau et al., 2021). Researchers measured topographical features as the corresponding appearance of the stimuli (e.g., “G” and “C”). Response topography was measured according to the required number of syllables and phonemic production associated with the correct response (e.g., nail and tail). After pairing potential stimuli, researchers assigned one of each stimulus pair to each training condition. For example, if “nail” and “tail” were paired as potential stimuli, researchers assigned “nail” to the DTI with IF condition and “tail” to the DTI condition. This process for equating stimuli was repeated across all sets of stimuli until both training conditions consisted of four primary targets and untrained conditions consisted of four secondary targets. All participants received training on a single set of assigned targets in each condition, except Jose, who received training on two sets of assigned primary targets in training conditions and one set of assigned secondary targets in the probe condition. Table 2 lists target stimuli and responses for primary and secondary targets.
General Procedures
Instructors conducted sessions with the participants one to three times per week. During each Zoom meeting, participants experienced at least one session of all conditions (described below). The DTI and DTI with IF conditions were presented in a randomized order. Instructors conducted probes every third session—after the participant received one exposure to both the DTI and DTI with IF conditions. Lasted for 1 to 5 min, and a 1 to 10 min break followed. During sessions with primary targets, instructors presented the four primary targets three times in a semi-randomized order, yielding a total of 12 trials. During probes, instructors presented all eight secondary targets (i.e., four in the DTI condition and four in the DTI with IF condition) twice in a semi-randomized order, yielding 16 trials.
At the start of each Zoom call, caregivers conducted an informal preference assessment of available edible items. Each preference assessment consisted of caregivers identifying available preferred edible items (e.g., mini chocolate chips), vocally listing various preferred edible items that were available to the child, and the child selecting one item. Researchers requested that caregivers provide small edible items because we anticipated that participants would take less time to consume smaller items. Additionally, researchers asked caregivers to provide edible reinforcers to ensure that caregivers would not require additional training on reinforcement systems (e.g., token economies) and participants would not be required to relinquish access to the preferred item multiple times throughout the session (e.g., accessing the item for 20 s and then handing the item to the caregiver).
Prior to implementing treatment sessions, researchers directed each caregiver to provide edible items if the participant emitted a correct response. Throughout treatment sessions, caregivers provided the selected item as the contingent consequence for correct responses.
Baseline and Secondary Target Probes
Instructors initiated each trial by presenting the target and the instruction (e.g., the screen displayed the letter, “A,” and the instructor said, “What sound?”). Instructors then allowed up to 5 s for the participant to respond, and no programmed consequences were delivered contingent on incorrect responding. In addition, instructors provided noncontingent, non-specific praise (e.g., “Thanks for sitting.”) in between every three to four trials. Finally, participants received contingent access to preferred video clips for completing every third session (i.e., probe sessions).
General Treatment Conditions
Instructors initiated trials by presenting the primary target and the instruction (see the top panel of Fig. 1). Then instructors allowed up to 5 s for the participant to respond. If the participant correctly responded (i.e., engaged in the primary target response), the participant received praise paired with a preferred edible item. If the participant responded incorrectly or did not respond to the presentation of the primary target, instructors removed the stimulus and provided feedback (e.g., the screen displayed “Try again” and the instructor said, “Try again.”). Then instructors re-presented the trial and provided an immediate (i.e., 0-s prompt delay) echoic prompt. If the participant incorrectly echoed the response, instructors continued to re-present the trial with the corresponding prompt until the participant correctly echoed the response. After a correct prompted response, instructors re-presented the original training trial without an echoic prompt. Instructors repeated the error correction sequence if the participant erred in the re-presented trial. When the participant correctly responded, the participant received praise paired with a preferred edible item.
DTI
During the consequent event, instructors presented the secondary target (e.g., a picture of a cow) and withheld the presentation of the secondary target response (see the top panel of Fig. 1). Caregivers did not orient participants toward the presentation of the secondary target, and instructors did not provide consequences if participants manded for information that related to the secondary target or engaged in a tact of the secondary target. Researchers included this presentation of the secondary target to control for simple exposures to the stimulus as a potential controlling variable. Secondary targets were presented an equal number of times each session, and stimulus presentation occurred in a semi-randomized order.
DTI with IF
Following the delivery of the preferred edible item, instructors presented a secondary target (e.g., a picture of a dog) and presented the secondary target response (e.g., “A dog is an animal” [see the bottom panel of Fig. 1]). Similar to DTI procedures, caregivers did not orient participants toward the presentation of the secondary target, and instructors did not provide consequences for manding or tacts of the secondary target.
Results
Figures 2 and 3 display the data for all participants. Figure 2 depicts the percentage of correct independent primary target responses emitted during each training condition. Participants did not emit correct responses during baseline. After implementing treatment, Jose’s responding met the mastery criteria for both training conditions in four sessions. In a second baseline, Jose did not emit correct responses to additional stimulus sets. After implementing treatment, Jose’s responding met the mastery criteria for both training conditions in three sessions. Elmer’s responding met the mastery criteria for the DTI condition in nine training sessions and the mastery criteria for the DTI with IF condition in 19 training sessions. Sabrina’s responding met the mastery criteria for both training conditions in five sessions. Stacy’s responding met the mastery criteria for the DTI condition in five training sessions and eight training sessions for the DTI with IF condition.
Figure 3 depicts the percentage of correct independent secondary target responses emitted during probe sessions. Jose did not respond correctly to stimuli assigned to either training condition throughout all probe sessions. Elmer did not emit a correct response for stimuli assigned to either training condition during baseline. Following the training, Elmer’s percentage of correct responding to secondary targets assigned to the DTI with IF condition initially increased to 37.5% (range, 12.5–50%), and responding to secondary targets assigned to the DTI condition remained at 0%. After eight probe sessions, Elmer’s level for correct responding decreased to 25% (range, 12.5–37.5%). Sabrina did not respond correctly to secondary targets assigned to either training condition throughout all probe sessions. Stacy emitted one correct response during the second baseline probe for a secondary target assigned to the DTI condition. Stacy did not emit a correct response for secondary targets assigned to either training condition for the remaining baseline probes. After training, Stacy’s percentage of correct responding to secondary targets assigned to the DTI with IF condition initially increased to 50% and remained between 87.5 and 100% (an outlier of 0% occurred on the fourth session). After three training sessions, the percentage of correct responding to secondary targets assigned to the DTI condition initially increased to 25% and decreased to 0% after three training sessions.
Discussion
The current findings suggest that DTI and DTI with IF appear to be efficacious approaches for teaching primary targets via telehealth. Two participants met the mastery criteria for the primary targets after the same number of DTI and DTI with IF training sessions. Elmer met the mastery criteria for the primary targets in the DTI condition 10 sessions before meeting the mastery criterion in the DTI with IF condition, and Stacy met the mastery criteria for the DTI condition three sessions before meeting the mastery criterion in the DTI with IF condition.
With regard to our second aim, our results showed that two of the four participants demonstrated some correct responding to secondary targets assigned to the DTI with IF condition. Elmer’s had low but variable levels of correct responding to secondary targets in the DTI with IF training. Stacy initially displayed high but variable levels of correct responding for secondary targets assigned to the DTI with IF condition and two sessions with some correct responses to secondary targets assigned to the DTI condition. However, during the final four probes, Stacy demonstrated the acquisition of secondary targets assigned to the DTI with IF condition. The variability in acquiring secondary targets across participants is consistent with previous research that evaluated novel targets (Brosh et al., 2018; Cariveau et al., 2022) but inconsistent with other studies that demonstrated higher levels of correct responding to secondary target or mastery of secondary, novel targets (Nottingham et al., 2017; Schnell et al., 2018). Overall, the results indicate that DTI with IF (a) did not hinder the acquisition of the primary targets and (b) may promote some acquisition of secondary targets in some participants. Given that the inclusion of IF is relatively easy for instructors and may not add a significant amount of instructional time, the potential benefits of its inclusion warrant consideration.
Two participants, Jose and Sabrina, did not display any acquisition of the secondary targets. As previously noted, both participants met all inclusion criteria (e.g., attend to digital materials, produce a vocal tact). Additionally, participants did not require procedural adaptations to address any barriers for acquisition (e.g., challenging behavior, low frequency of attending to the screen). One possible explanation for Jose and Sabrina’s responding during probe sessions might relate to each participant’s performance during telehealth sessions. Specifically, previous research suggests that current repertoires (e.g., echoic responses to demonstrations of secondary target responses) may influence the acquisition of secondary targets (Haq et al., 2017; Leaf et al., 2017). However, researchers did not measure echoic responses when instructors stated the secondary target responses in the current study. It is possible that echoic responding that was observed during in-person training (see VB-MAPP scores on Table 1) may not reflect each participant’s performance during telehealth sessions. For instance, the stimulus control observed during in-person sessions may not have generalized to the telehealth sessions. Alternatively, contextual factors within telehealth settings (e.g., slight time delays, auditory and visual qualities) may have contributed to the overall poor acquisition of secondary targets. Future research on IF procedures should consider the potential influence prerequisite skills have on acquiring secondary targets. Additionally, future research may consider evaluating how responding during in-person sessions (e.g., echoic responses to demonstrations of the secondary target responses) influences the acquisition of secondary targets when sessions are conducted via telehealth.
A number of limitations in the present study are worth noting. First, several considerations regarding counterbalancing targets might be necessary to ensure that all stimulus sets were of equal difficulty. During the current study, we assigned stimuli to conditions based on topographical features and response topography. However, our procedures for set assignment did not consider cultural exposure and familiarity with stimuli. For example, letter sounds “f” and “v” were assigned to separate stimulus sets because of the response topography associated with each stimulus. However, the participant may have encountered the “f” stimulus in their natural environment more often than the “v” stimulus. The difference in how frequently participants contact the stimuli in their natural environment may influence the participant’s responding during teaching sessions. Future research may consider assigning stimuli based on cultural exposures.
A second potential limitation related to procedural considerations when presenting demonstrations of the secondary target (i.e., IF procedures) and secondary target probes. In the current study, three of the four participants received demonstrations of the secondary targets that differed from the antecedent presented during the probe trials. For example, “A moose lives in a forest,” was the second target demonstrated, whereas the antecedent for the probe trial was, “Where does a moose live?” Of the three participants exposed to this variation, two participants demonstrated some acquisition of secondary targets. In contrast, Jose’s stimulus and response relations were reversed in secondary target demonstrations and probe trials. For example, the secondary target was, “A moose lives in a forest,” whereas the instructor said, “What animal lives in the forest?” during the probe trials. Jose did not engage in any correct response to secondary targets. Based on the results, one might question if the similarity of antecedent and responses arranged in secondary target demonstrations and probe trials influences acquisition of the secondary targets (Aguirre et al., 2019). Previous studies have considered similar points with respect to emergent behavior (Delmolino et la., 2013; Petursdottir et al., 2008; Shillingsburg et al., 2018). Future researchers should evaluate the extent to which the antecedent and response arrangements affect the measured acquisition of secondary targets.
Another potential limitation of our study relates to the experimental design. Our study used an AATD embedded in a multiple baseline design to compare the effects of two teaching procedures. Previously, Cariveau and Fetzner (2022) highlighted that this experimental design fails to detect multiple-treatment interference and historical threats to internal validity. Specifically, the prolonged exposure to extended baseline in the staggered panels (e.g., Sabrina’s baseline) may be unnecessary because it does not allow for the detection of these important threats, particularly when the participants were not in the same clinic during this period. To address this, future research may consider using an AATD that includes a control condition.
The final highlighted limitation of the study relates to the dosage of the intervention. Specifically, instructors conducted research sessions based on each participant’s availability. In some cases, participants had limited availability which resulted in research sessions being conducted one to three times per week. As a result of this low frequency of research sessions, participants may have received too few exposures to stimuli to increase responding (Haq & Kodak, 2015). Future research may consider if a higher dosage of treatment may influence the effectiveness of IF procedures.
Researchers may consider further evaluations of IF procedures delivered through telehealth services. Specifically, future research comparing the efficiency of DTI and DTI with IF may consider evaluating multiple efficiency measures (see Kodak et al., 2016). A second area of future research relates to the schedule of reinforcement during IF probes. In the current study, researchers withheld contingent access to preferred stimuli to ensure that the acquisition of secondary targets was related to the presence of IF procedures and not due to differential reinforcement of correct responding. Frampton and Shillingsburg (2020) also withheld praise contingent upon correct responding during probes. Future research may consider comparing the effects of different reinforcement preparations on the emergence of secondary targets (see LeBlanc et al., 2003)
Despite these limitations, the results of this study contribute to the growing literature on IF. Specifically, our data provide preliminary evidence that DTI procedures delivered via telehealth may produce positive clinical outcomes for learners with ASD. Additionally, the potential benefits associated with both telehealth delivery and IF appear to warrant future research.
Data Availability
The datasets generated and analyzed during the current study are available from the corresponding author on reasonable request.
References
Aguirre, A. A., LeBlanc, L. A., Reavis, A., Shillingsburg, A. M., Delfs, C. H., Miltenberger, C. A., & Symer, K. B. (2019). Evaluating the effects of similar and distinct discriminative stimuli during auditory conditional discrimination training with children with autism. The Analysis of Verbal Behavior, 35(1), 21–38. https://doi.org/10.1007/s40616-019-00111-3
Albarran, S. A., & Sandbank, M. P. (2019). Teaching non-target information to children with disabilities: An examination of instructive feedback literature. Journal of Behavioral Education, 28(1), 107–140. https://doi.org/10.1007/s10864-018-9301-3
Brosh, C. R., Root, J. R., Saunders, A. F., Spooner, F., & Fisher, L. B. (2018). Embedding literacy in mathematics problem solving instruction for learners with intellectual and developmental disability. Inclusion, 6(2), 81–96. https://doi.org/10.1352/2326-6988-6.2.81
Cariveau, T., Batchelder, S., Ball, S., & La Cruz Montilla, A. (2021). Review of methods to equate target sets in the adapted alternating treatments design. Behavior Modification, 45(5), 695–714. https://doi.org/10.1177/0145445520903049
Cariveau, T., Brown, A., Platt, D., Ellington, P., & Hurtado, R. (2022). An evaluation of instructive feedback during mastered demands. The Analysis of Verbal Behavior, 1–11. https://doi.org/10.1007/s40616-022-00174-9
Cariveau, T., & Fetzner, D. (2022). Experimental control in the adapted alternating treatments design: A review of procedures and outcomes. Behavioral Interventions, 37(3), 805–818. https://doi.org/10.1002/bin.1865
Centers for Medicare and Medicaid Services. (2018). Information on Medicare Telehealth. Retrieved July 6, 2022 from https://www.cms.gov/About-CMS/Agency-Information/OMH/Downloads/Information-on-Medicare-Telehealth-Report.pdf
Delmolino, L., Hansford, A. P., Bamond, M. J., Fiske, K. E., & LaRue, R. H. (2013). The use of instructive feedback for teaching language skills to children with autism. Research in Autism Spectrum Disorders, 7(6), 648–661. https://doi.org/10.1016/j.rasd.2013.02.015
Ellison, K. S., Guidry, J., Picou, P., Adenuga, P., & Davis, T. E. (2021). Telehealth and autism prior to and in the age of COVID-19: A systematic and critical review of the last decade. Clinical Child and Family Psychology Review, 24(3), 599–630. https://doi.org/10.1007/s10567-021-00358-0
Ferguson, J. L., Majeski, M. J., McEachin, J., Leaf, R., Cihon, J. H., & Leaf, J. B. (2020). Evaluating discrete trial teaching with instructive feedback delivered in a dyad arrangement via telehealth. Journal of Applied Behavior Analysis, 53(4), 1876–1888. https://doi.org/10.1002/jaba.773
Frampton, S. E., & Shillingsburg, M. A. (2020). Promoting the development of verbal responses using instructive feedback. Journal of Applied Behavior Analysis, 53(2), 1029–1041. https://doi.org/10.1002/jaba.659
Gavidia, V. L., Bergmann, S., & Rader, K. A. (2022). The use of instructive feedback to promote emergent tact and intraverbal control: A replication. The Analysis of Verbal Behavior, 1–26. https://doi.org/10.1007/s40616-022-00171
Google LLC. (2006). Google Slides [Web application]. Retrieved from https://google.com/slides/about/
Grow, L. L., Kodak, T., & Clements, A. (2017). An evaluation of instructive feedback to teach play behavior to a child with autism spectrum disorder. Behavior Analysis in Practice, 10(3), 313–317. https://doi.org/10.1007/s40617-016-0153-9
Haq, S. S., & Kodak, T. (2015). Evaluating the effects of massed and distributed practice on acquisition and maintenance of tacts and textual behavior with typically developing children. Journal of Applied Behavior Analysis, 48(1), 85–95. https://doi.org/10.1002/jaba.178
Haq, S. S., Zemantic, P. K., Kodak, T., LeBlanc, B., & Ruppert, T. E. (2017). Examination of variables that affect the efficacy of instructive feedback. Behavioral Interventions, 32(3), 206–216. https://doi.org/10.1002/bin.1470
Hyland, K. A., McDonald, J. B., Verzijl, C. L., Faraci, D. C., Calixte-Civil, P. F., Gorey, C. M., & Verona, E. (2022). Telehealth for dialectical behavioral therapy: A commentary on the experience of a rapid transition to virtual delivery of DBT. Cognitive and Behavioral Practice, 29(2), 367–380. https://doi.org/10.1016/j.cbpra.2021.02.006
Kodak, T., Campbell, V., Bergmann, S., LeBlanc, B., Kurtz-Nelson, E., Cariveau, T., & Mahon, J. (2016). Examination of efficacious, efficient, and socially valid error-correction procedures to teach sight words and prepositions to children with autism spectrum disorder. Journal of Applied Behavior Analysis, 49(3), 532–547. https://doi.org/10.1002/jaba.310
Leaf, J. B., Cihon, J. H., Alcalay, A., Mitchell, E., Townley-Cochran, D., Miller, K., & McEachin, J. (2017). Instructive feedback embedded within group instruction for children diagnosed with autism spectrum disorder. Journal of Applied Behavior Analysis, 50(2), 304–316.
LeBlanc, L. A., Miguel, C. F., Cummings, A. R., Goldsmith, T. R., & Carr, J. E. (2003). The effects of three stimulus-equivalence testing conditions on emergent US geography relations of children diagnosed with autism. Behavioral Interventions: Theory & Practice in Residential & Community-Based Clinical Programs, 18(4), 279–289. https://doi.org/10.1002/bin.144
Loughrey, T. O., Betz, A. M., Majdalany, L. M., & Nicholson, K. (2014). Using instructive feedback to teach category names to children with autism. Journal of Applied Behavior Analysis, 47(2), 425–430. https://doi.org/10.1002/jaba.123
Love, J. R., Carr, J. E., Almason, S. M., & Petursdottir, A. I. (2009). Early and intensive behavioral intervention for autism: A survey of clinical practices. Research in Autism Spectrum Disorders, 3(2), 421–428. https://doi.org/10.1016/j.rasd.2008.08.008
Mattson, S. L., Higbee, T. S., Aguilar, J., Nichols, B., Campbell, V. E., Nix, L. D., et al. (2020). Creating and sharing digital ABA instructional activities: A practical tutorial. Behavior Analysis in Practice, 13(4), 772–798. https://doi.org/10.1007/s40617-020-00440-z
Nickelson, D. W. (1998). Telehealth and the evolving health care system: Strategic opportunities for professional psychology. Professional Psychology: Research and Practice, 29(6), 527–535. https://doi.org/10.1037/0735-7028.29.6.527
Nohelty, K., Bradford, C. B., Hirschfeld, L., Miyake, C. J., & Novack, M. N. (2021). Effectiveness of telehealth direct therapy for individuals with autism spectrum disorder. Behavior Analysis in Practice, 1–16. https://doi.org/10.1007/s40617-021-00603-6
Nottingham, C. L., Vladescu, J. C., DeBar, R. M., Deshais, M., & DeQuinzio, J. (2020). The influence of instructive feedback presentation schedule: A replication with children with autism spectrum disorder. Journal of Applied Behavior Analysis, 53(4), 2287–2302. https://doi.org/10.1002/jaba.706
Nottingham, C. L., Vladescu, J. C., Kodak, T., & Kisamore, A. N. (2017). Incorporating multiple secondary targets into learning trials for individuals with autism spectrum disorder. Journal of Applied Behavior Analysis, 50(3), 653–661.https://doi.org/10.1002/jaba.396
Nottingham, C. L., Vladescu, J. C., & Kodak, T. M. (2015). Incorporating additional targets into learning trials for individuals with autism spectrum disorder. Journal of Applied Behavior Analysis, 48(1), 227–232. https://doi.org/10.1002/jaba.179
Petursdottir, A. I., ÓLafsdóttir, A. R., & Aradóttir, B. (2008). The effects of tact and listener training on the emergence of bidirectional intraverbal relations. Journal of Applied Behavior Analysis, 41(3), 411–415. https://doi.org/10.1901/jaba.2008.41-411
Pollard, J. S., LeBlanc, L. A., Griffin, C. A., & Baker, J. M. (2021). The effects of transition to technician-delivered telehealth ABA treatment during the COVID-19 crisis: A preliminary analysis. Journal of Applied Behavior Analysis, 54, 87–102. https://doi.org/10.1002/jaba.803
Reichow, B., & Wolery, M. (2011). Comparison of progressive prompt delay with and without instructive feedback. Journal of Applied Behavior Analysis, 44(2), 327–340. https://doi.org/10.1901/jaba.2011.44-327
Schnell, L. K., Vladescu, J. C., Kodak, T., & Nottingham, C. L. (2018). Comparing procedures on the acquisition and generalization of tacts for children with autism spectrum disorder. Journal of Applied Behavior Analysis, 51(4), 769–783. https://doi.org/10.1002/jaba.480
Shillingsburg, M. A., Frampton, S. E., Cleveland, S. A., & Cariveau, T. (2018). A clinical application of procedures to promote the emergence of untrained intraverbal relations with children with autism. Learning and Motivation, 62, 51–66. https://doi.org/10.1016/j.lmot.2017.02.003
Sindelar, P. T., Rosenberg, M. S., & Wilson, R. J. (1985). An adapted alternating treatments design for instructional research. Education and Treatment of Children, 67–76. https://doi.org/10.1007/s10864-018-9301-3
Smith, T. (2001). Discrete trial training in the treatment of autism. Focus on Autism and Other Developmental Disabilities, 16(2), 86–92 https://www-jstor-org.dist.lib.usu.edu/stable/42898888
Sundberg, M. L. (2008). Verbal behavior milestones assessment and placement program. Concord, CA: AVB.
Tullis, C. A., Frampton, S. E., Delfs, C. H., Greene, K., & Reed, S. (2021). The effects of instructive feedback and stimulus equivalence procedures on group instructional outcomes. Journal of Behavioral Education, 30(1), 1–21. https://doi.org/10.1007/s10864-019-09349-2
Tullis, C. A., Frampton, S. E., Delfs, C. H., & Shillingsburg, M. A. (2017). Teaching problem explanations using instructive feedback. The Analysis of Verbal Behavior, 33(1), 64–79. https://doi.org/10.1007/s40616-016-0075-1
Tullis, C. A., Gibbs, A. R., Priester, J., & Tillem, A. (2022). Emergence of verbal responses using instructive feedback: A replication and extension. Behavioral Interventions, 37(2), 271–289. https://doi.org/10.1002/bin.1836
Vladescu, J. C., & Kodak, T. M. (2013). Increasing instructional efficiency by presenting additional stimuli in learning trials for children with autism spectrum disorders. Journal of Applied Behavior Analysis, 46(4), 805–816. https://doi.org/10.1002/jaba.70
Werts, M. G., Wolery, M., Holcombe, A., & Gast, D. L. (1995). Instructive feedback: Review of parameters and effects. Journal of Behavioral Education, 5(1), 55–75. https://doi.org/10.1007/BF02110214
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Campbell, V.E., Higbee, T.S., Osos, J.A. et al. A Comparison of Telehealth-Based Instruction with or without Instructive Feedback. Analysis Verbal Behav (2023). https://doi.org/10.1007/s40616-023-00185-0
Accepted:
Published:
DOI: https://doi.org/10.1007/s40616-023-00185-0