Abstract
Paraprofessionals play a critical role in the instruction of students with disabilities and yet they often receive inadequate training in using evidence-based teaching strategies. A promising avenue for improving paraprofessional training is distant bug-in-ear coaching (BIE), where a paraprofessional receives in-the-moment coaching on a teaching strategy from a coach at a different location. This study examined a BIE coaching package to support paraprofessionals in using incidental teaching for teaching self-advocacy skills to students with disabilities. The package included an initial individual didactic teaching session followed by distance BIE coaching. A multiple-baseline across participants design was used to assess the impact of the intervention on both the skills of the paraprofessionals and on student acquisition of self-advocacy statements. BIE coaching was associated with increases in both the accuracy and rate of incidental teaching trials and with use of self-advocacy statements by the students with disabilities.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
Introduction
Paraprofessionals have long played an important supportive role in schools, and they play an increasingly large role in the instruction of children with disabilities (McDonnell and Jameson 2014). During the 2014–2015 academic year, there were nearly 440,000 special education paraprofessionals working in the USA and outlying areas compared to roughly 350,000 certified special educators (U.S. Department of Education 2016). Paraprofessionals report responsibilities for large-group, small-group, and one-on-one instruction in schools with children of all ages and disability categories (Carter et al. 2009). These are significant responsibilities and are contingent upon appropriate training and supervision.
Under federal law, paraprofessionals are authorized to assist in the provision of special education and related services (IDEA 2004), yet there is doubt that paraprofessionals receive adequate training and supervision for their responsibilities (Giangreco et al. 2010). In a systematic literature review examining the outcomes of paraprofessional-implemented educational practices and professional development strategies, Brock and Carter (2013) identified only 13 studies in which paraprofessionals received any form of professional development on instructional practices. It is unclear the extent to which paraprofessionals are participating in professional development that is active, meaningful, and focused on the content they need in their daily practice (Garet et al. 2001).
Coaching with performance feedback has emerged as a particularly promising evidence-based practice for improving desired teaching behaviors (Cornelius and Nagro 2014; Fallon et al. 2015). Typical coaching with performance feedback interventions for paraprofessionals has ranged from brief verbal performance feedback on implementation of target strategies (Kim et al. 2017; Maggin et al. 2017) to 10-min coaching conversations after an observation (Ledford et al. 2017). Coaching conversations included goal review (Ledford et al. 2017), modeling (Ledford et al. 2017; Maggin et al. 2017) and positive and corrective feedback (Kim et al. 2017; Ledford et al. 2017; Maggin et al. 2017). While coaching sessions were brief (5–10 min) across all studies, even brief coaching has been associated with stronger effects than traditional strategies such as didactic training and video modeling (Brock and Carter 2013).
Innovations in Coaching for Paraprofessionals: Bug-in-Ear Coaching
Most paraprofessional coaching studies to date have used traditional coaching models, requiring the coach to be physically present with the paraprofessional. In recent years, technology has allowed new opportunities for coaching. One innovation, bug-in-ear (BIE) coaching, holds particular promise for paraprofessional training. BIE coaching provides real-time, immediate feedback and support, while an adult works with a student or group of students. In most examples of BIE coaching, the coachee wears a small wireless device in the ear. A coach watches the coachee’s instruction from a different area of the room (e.g., Goodman et al. 2008; Ottley and Hanline 2014; Scheeler et al. 2018) or from a remote location via video conferencing systems such as Skype (e.g., Rock et al. 2014). Throughout the observation, the coach provides immediate feedback, suggestions, or prompts consistent with recommendations for effective performance feedback (Scheeler et al. 2004).
BIE coaching has been used to increase pre-service behavior analysts’ implementation of functional communication training (Artman-Meeker et al. 2017), pre-service teachers’ use of evidence-based practices (Rock et al. 2014), general education teachers’ support of student self-monitoring (Owens et al. 2019), and families’ use of communication strategies (Wacker et al. 2013). BIE coaching requires little to no time outside of the classroom, is easy to learn, and is unobtrusive during instruction (Scheeler et al. 2018).
Despite this promise, only one study to date (Scheeler et al. 2018) has used BIE coaching to support paraprofessionals. In this study, the coach (the teacher) was in the classroom with the paraprofessional receiving support. Distance BIE coaching, where the coach is at a remote location from the coachee, may hold particular promise for training paraprofessionals. By offering high-quality, in-the-moment coaching to paraprofessionals without the time and cost of travel, a coach such as a district educational or behavior specialist could substantially increase the number of paraprofessionals supported. To our knowledge, no studies have used distance BIE coaching to coach paraprofessionals.
Incidental Teaching and BIE Coaching
Incidental teaching (IT) is an evidence-based practice (Wong et al. 2014) in which adults modify a learner’s natural environment to enhance the learner’s motivation to use a target skill (Hart and Risley 1982; Fenske et al. 2001) and then use prompting and naturally occurring reinforcement to increase correct responding (McGee et al. 1985). A substantial body of research has demonstrated the effectiveness of incidental teaching across a wide age range of learners and settings (e.g., Charlop-Christy and Carpenter 2000; Hart and Risley 1968, 1975; Kroeger and Nelson 2006; McGee et al. 1999; Schepis et al. 1982).
Incidental teaching is a highly individualized teaching method requiring rapid, sophisticated decisions about a child’s motivation and natural reinforcement opportunities. BIE coaching may be an ideal fit for such a complex intervention. BIE coaches could help paraprofessionals create appropriate opportunities for practice, provide well-timed prompting and reinforcement, and adjust teaching as necessary based on student response.
Self-Advocacy as a Target Skill
Self-advocacy is a strengths-based outcome rooted in the US civil rights and disability rights movements (Test et al. 2005). It has been identified as a key component of self-determination, the ability to guide meaningful choices about one’s own life (Field et al. 1998). A growing body of literature suggests a positive association between self-determination and outcomes into adulthood for individuals with disabilities (Powers et al. 2012; Shogren et al. 2015; Wehmeyer and Palmer 2003). Self-advocacy is a critical aspect of self-determination, focusing on an individual’s ability to communicate one’s wants and needs in acceptable ways (Wood et al. 2005). Self-advocacy has been identified as an important target for instruction for students with disabilities (Izzo and Lamb 2002).
The incidental teaching literature has established that communication skills for individuals with disabilities are ideally taught during naturally occurring circumstances (Miranda-Linne and Melin 1992). Given the daily instructional contact between students with disabilities and paraprofessionals, paraprofessionals are in a unique position to teach students self-advocacy communications in authentic ways. Incidental teaching offers paraprofessionals the opportunity to embed meaningful instruction on self-advocacy skills during typical curricular experiences.
The purpose of the present study was to examine the effects of a BIE coaching package on paraprofessional use of incidental teaching to increase self-advocacy statements for students with developmental disabilities in K-12 classroom settings.
Two questions guided this study:
-
Research Question 1: Does a BIE coaching package increase paraprofessionals’ implementation of incidental teaching as an instructional strategy for teaching self-advocacy statements to students with disabilities?
-
Research Question 2: Does paraprofessional implementation of incidental teaching of self-advocacy statements increase student use of target statements?
Method
Setting and Participants
This study was conducted at a private K-12 school located in the US Pacific Northwest, serving students with and without disabilities. Focal participants consisted of four adult-student dyads. The focus on self-advocacy skills was decided after collaboration with the school clinical director, who communicated that self-advocacy was a goal for all children in the school and that the school would appreciate help in how to best teach those skills. To identify potential participants, the school clinical director consulted with teachers and provided the researchers with a list of possible paraprofessional/student dyads. The students on the list were all identified as having an Individualized Education Plan (IEP) and current IEP goals around self-advocacy statements. The paraprofessionals all worked directly with one of the target students and were identified as likely to benefit from coaching on incidental teaching. The final dyads were chosen based on scheduling matches with coach schedules. Researchers collaborated with the classroom teachers of these dyads in order to identify the best activities and settings for the coaching sessions.
Dyad 1: Yara and Yolanda
Yara was a Caucasian female who had worked at the school for 4 years as a paraprofessional. She had a bachelor’s degree and no previous training in incidental teaching. Yolanda was a 13-year-old, 7th-grade female student with diagnoses of autism and anxiety disorder. Yolanda received 1:1 support from a paraprofessional for half the school day in her inclusive classroom. Yolanda’s verbal language was age-appropriate; she spoke in full sentences and could ask and answer questions. Yolanda had three advocacy statements that were targeted in this study: “I don’t understand,” “Can you give me an example?”, and “Can you tell me what I did well?” All three statements were included in a self-advocacy goal on Yolanda’s IEP. Sessions for this dyad took place during Yolanda’s writing class, a time when Yolanda received regular 1:1 support. Generalization sessions for Yolanda occurred in the same classroom later in the day during math class.
Dyad 2: Brianna and Bonnie
Brianna, a Caucasian female paraprofessional with a bachelor’s degree, had worked at the school for 1 year and had no previous training in incidental teaching. Bonnie was a 16-year-old, high school female with an educational diagnosis of autism. She received 1:1 paraprofessional support through the school day. Bonnie had very little verbal communication and primarily used an iPad with the communication app ProloQuo2Go to communicate. The target self-advocacy phrase for her in this study, taken from her IEP, was asking “Where is it?” when she couldn’t find a desired object. Brianna and Bonnie’s sessions were held in Bonnie’s inclusive classroom during a work period where various functional life skills were targeted. Generalization probes were collected in the school kitchen where Bonnie was learning how to make a snack.
Dyad 3: Gary and Gretta
Gary, an Asian male paraprofessional, had worked at the school for 4 years. Gary had a bachelor’s degree and no previous training in incidental teaching. Gretta was an 11-year-old, 5th-grade female student with an educational diagnosis of autism. She received 1:1 paraprofessional support throughout the school day in her inclusive classroom. Gretta had limited verbal communication, but she communicated orally using 1-3 word phrases. Her targeted advocacy statement, taken from her IEP, was “I need help,” communicated verbally. Gary and Gretta’s sessions took place during Gretta’s math class. Generalization sessions were conducted in the same classroom at a different time of day with different activities: art and reading.
Dyad 4: Polly and Pietra
Polly was a Caucasian female paraprofessional with a bachelor’s degree, who had worked at the school for 9 months. Polly, like the other paraprofessionals, had no previous training in incidental teaching. Pietra was a 5-year-old female in kindergarten. Pietra was reported to have mixed receptive and expressive language disorder and speech articulation developmental disorder. She received 1:1 paraprofessional support in her inclusive classroom during times identified as areas of need, such as math. She communicated in full sentences. Her targeted advocacy statement for this study was “This is too hard,” a phrase identified as a self-advocacy goal on Pietra’s IEP. Polly and Pietra’s sessions took place during pull-out math sessions. The pull-out sessions occurred in Pietra’s classroom, while the other students engaged in a group math activity. Due to the end of school year, no generalization probes occurred for Polly and Pietra.
Coaches
Each team was assigned a coach from the research team to deliver the BIE coaching. Coaches were second- and third-year doctoral students in special education. All four coaches were female. Three of the coaches were also board certified behavior analysts and had been coaching teachers and behavior technicians on behavioral strategies through a university applied behavior analysis program for at least 1 year. The fourth student had been a research assistant on a previous BIE study. Coaching assignments were made based on the alignment of coach availability with the requested session times of the dyads.
Materials
Each paraprofessional wore a wireless Bose Bluetooth Head Set Series 2 earpiece (BIE device) during each session, allowing two-way communication with the coach. The BIE device connected via Bluetooth with a WiFi-enabled iPod® Touch, provided by the research team. The iPod Touch was mounted on a C-series SWIVL Robot platform and provided the video feed for the BIE coaching as well as the audio connection between the paraprofessional and coach. The paraprofessional wore a SWIVL marker on a lanyard around his or her neck which enabled the camera on the SWIVL device to follow the paraprofessional if he or she moved around the classroom.
BIE coaching sessions were conducted over Zoom, a secure video conferencing technology. All Zoom connections were initiated from a university office approximately 1.5 miles from the participants’ school and used the university’s encrypted wireless network. Before each session, the coach initiated a Zoom meeting on a MacBook Air® laptop computer by sending a Zoom invitation link to the appropriate iPod using iMessage. The paraprofessionals accepted the Zoom meeting invitation on the iPod Touch; calls were connected using the school’s wireless internet. Three of the four classrooms had a dedicated internet router in the classroom; only Yara and Yolanda’s classroom did not have a dedicated router.
All meetings were recorded and saved to the laptop computers using Zoom’s built-in recording software. Videos were collected and stored in compliance with federal laws. Additionally, all participants (or in the case of the students, their parents) signed consent and release forms authorizing the use of Zoom and the use of video recording for research purposes.
Dependent Measures
Paraprofessional Behavior
There were three dependent measures for paraprofessional behavior: percentage correct of incidental teaching components per session, rate of incidental teaching trials implemented in a session, and number of trials with 100% correct components.
Percent Correct IT Components
Each incidental teaching trial had 2–5 possible components. Components included one motivating operation, one to three prompts depending on the student’s behavior, and one response to the student’s behavior. A component was scored correct if a paraprofessional implemented it independently and correctly. To calculate the percentage of correctly implemented incidental teaching components per session, the number of correct components across all trials was divided by the number of expected components across all trials and multiplied by 100 to calculate a percentage correct.
Rate of IT Trials
Count data were collected on paraprofessionals’ incidental teaching trials of each student’s target skill per session and then was divided by session time to calculate a rate of incidental teaching trials.
Trials with 100% Correct Components
The number of trials with 100% correct components was counted per session. This count was used in determining when a paraprofessional had met mastery criteria for correct implementation of incidental teaching.
Student Behavior
There were two dependent variables for student behavior: rate of combined prompted and independent target self-advocacy statement(s) and rate of independent target self-advocacy statement(s) used by students per session.
Rate of Prompted and Independent Target Statements
Count data were collected on a student’s use of the target statement(s) per session, both prompted and unprompted. This was divided by session time to calculate rate of usage.
Rate of Independent Target Statements
The number of independent (unprompted) uses of the target statement for a student was divided by session time to calculate rate of independent target statements.
Technology Issues
All data were collected from video-recorded Zoom sessions. Technology issues occasionally occurred during sessions. These issues were addressed consistently during data coding. Periods in which both audio and video were frozen for more than 5 s were not coded and the time elapsed during the technology failure was recorded. If video slowed or froze, but audio continued and the coder could still discriminate what was happening, coding continued through the technology lapses; otherwise, these segments were not coded and the time lost was recorded. After the entire session had been coded, total time lost to technology was calculated and this total was subtracted from session length. Primary coder session times were used in reporting technology issues.
Social Validity
Paraprofessionals were asked to provide feedback on their experience with BIE coaching through an anonymous online survey. The survey consisted of 9 questions rated on a 5-point Likert-like scale and 4 open-ended questions on various aspects of the intervention.
Research Design
A concurrent multiple-baseline design across participants was used (Ledford and Gast 2018), with the start of intervention staggered across time for the four dyads. The goal of this design is to show behavior change in a participant when and only when the intervention is implemented. Intervention continued for each participant until mastery criteria of 5 trials at 100% fidelity over two consecutive days and at least one independent use of the target phrase by the child across 2 days was reached.
Procedures
Introduction to Equipment
Prior to baseline, two researchers led a 30-min group meeting with the four participating paraprofessionals. The researchers provided an overview of the study, introduced the BIE equipment, demonstrated how to set up the equipment, and provided each paraprofessional with a handout describing the steps for setting up the equipment. The target strategy (incidental teaching) and the focus of the intervention (self-advocacy) were not mentioned during this meeting.
During all study sessions, the paraprofessionals or their corresponding classroom teachers were expected to set up the BIE equipment. However, during most sessions a graduate student from the university was present to help with technology issues or to set up equipment if needed.
Baseline
During baseline, paraprofessionals were asked to interact with and provide instruction to the student as they normally would. Paraprofessionals were not told the specific behaviors being targeted by the study, although all paraprofessionals were familiar with their student’s IEP goals, which included the target phrases. Approximately 5 min before the start of each baseline session, the paraprofessional put on the BIE device and connected with the researcher on the iPod via the Zoom video conferencing technology. After greeting the paraprofessional, the researcher paused the recording until the paraprofessional indicated to start the teaching session. The researcher then resumed recording, but did not interact again with the paraprofessional until 20 min had elapsed. At this time, the researcher thanked the paraprofessional and ended the recording. No coaching was provided during baseline sessions.
BIE Teaching Package
Following baseline, a BIE teaching package was introduced. This was comprised of an initial 45 min training with the paraprofessional followed by BIE coaching.
Initial Training
After baseline, but before a participant dyad entered the coaching phase, the coach met individually with the paraprofessional for a 45-min introduction to incidental teaching. During this session, the coach shared a short PowerPoint presentation that included a definition of incidental teaching, a description of how it would help the paraprofessional and student, and an explanation of the components of incidental teaching. The coach reviewed the specific self-advocacy statement(s) the paraprofessional would be targeting. The coach and paraprofessional brainstormed motivational operations to promote target skills and agreed on an individualized prompt hierarchy. The coach and paraprofessional then role-played incidental teaching trials for the target behavior.
BIE Coaching
Because of several demonstrations in the literature that professional development without follow-up coaching is ineffective (Garet et al. 2001; Joyce and Showers 2002), we moved directly from the initial training to the coaching phase. During the coaching phase, coaches established a Zoom connection with the paraprofessionals approximately 5 min before the teaching session began. The coach greeted the paraprofessional and confirmed the incidental teaching plan for that session. The coach then paused the recording until the paraprofessional indicated to start the teaching session. The coach resumed recording and provided in-the-moment coaching to the paraprofessional via the BIE device for approximately 20 min. (Sessions varied slightly in length, based on the natural end of activities during the session.)
During coaching sessions, each paraprofessional used incidental teaching by presenting a motivating situation, prompting their student to use the target communication if necessary, and then naturally reinforcing the target statements. Paraprofessionals Yara, Gary, and Polly used the primary motivating operation of presenting difficult or unfamiliar work to evoke student target statements. Brianna placed learning materials out of sight as a motivating situation. Prompting strategies were individualized based on each student’s learning history and in consultation with paraprofessionals. Yara used a least-to-most prompting hierarchy across all sessions. Gary used a most-to-least prompting hierarchy for the first session and a least-to-most hierarchy for all further sessions. Polly used a most-to-least prompting hierarchy across all sessions.
Consistent with best practice in effective performance feedback, coaches were instructed to provide feedback that was immediate, specific, constructive, and purposeful (Cornelius and Nagro 2014; Scheeler et al. 2004). Following correct implementation of individual IT components and complete trials, coaches generally provided immediate, behavior-specific praise (e.g., “You gave your student reinforcement right away when she advocated for herself. That was great!”). Following incorrect implementation of IT components and trials, coaches generally provided constructive or suggestive feedback for how to deliver the IT trial correctly (e.g., “Next time, after you present the difficult work, try pausing before giving your student an immediate prompt.”). Coaches waited for a natural pause in communication between paraprofessionals and students before providing feedback. Occasionally, no natural pauses occurred and coaches either briefly interrupted the communication interaction to provide brief, general praise (e.g., “Nice job!”) or withheld feedback. All coaching feedback was purposefully targeted toward incidental teaching practices; no feedback was provided about unrelated teaching practices.
At the end of the coaching session, the coach thanked the paraprofessional for participation, provided a 1–2 sentence summary of the session, asked the paraprofessional if he or she had any questions, and reminded the paraprofessional that she would send a follow-up email. Due to the paraprofessionals’ classroom responsibilities, these discussions were brief. Within 24 h, the coach sent a follow-up email to the paraprofessional. The email followed a 5-step protocol based on Hemmeter et al. (2011).
Generalization
Probes were conducted for skill generalization at a different time of day and a different activity for each dyad. Classroom teachers were asked to identify a different activity in which the targeted self-advocacy skill would be useful. For Yara and Gary, the generalization probes were conducted at the same location, but at a different time of day and with a different activity. For Brianna, the generalization probes were conducted in a different setting with a different activity and at a different time of day. Due to the end of school year, no generalization probes occurred for Polly. Generalization session was conducted in the same manner as baseline sessions; no coaching was provided.
Maintenance
For three of the participants, maintenance probes were conducted to assess whether student and parabehaviors would continue in the absence of coaching. Maintenance probes were conducted in the same setting at the same time as the coaching sessions, but no BIE coaching was provided. For Yara and Brianna, two maintenance probes were collected, approximately 1 and 2 weeks after the coaching ended. For Gary, one maintenance probe was collected approximately 1 week after the end of intervention. Because of the end of the school year, no maintenance data were collected for Polly.
Interobserver Agreement and Treatment Fidelity
Interobserver agreement (IOA) was collected for a minimum of 25% of all sessions for each phase of the study for each participant. All coders were graduate students trained on the dependent variable coding system. To assess IOA, two coders independently scored randomly selected videos. IOA was calculated using the point-by-point agreement method with time-stamped counts whereby the number of agreements that began within a 20 s window was divided by the sum of agreements and disagreements and then multiplied by 100 to obtain a percentage (Ledford and Gast 2018). Each component of the incidental teaching trial was scored and included in the IOA calculations. IOA averaged 94% (range 60–100%) across all dyads and conditions. The IOA means and ranges within the phases of each dyad are presented in Table 1. Analysis of video footage of the single session in which Brianna’s session received an IOA score of 60% revealed that technology issues were present throughout the duration of the session; the SWIVL platform holding the iPod Touch repeatedly pivoted out of view of the dyad.
Treatment fidelity data were also collected on coach behavior. A checklist was used to code the following coach behaviors: (1) at start of session, coach confirms incidental teaching goal, including level of prompting and proposed motivational setups. (2) After coaching, coach thanks participant for observation and provides 1–2 sentence summary of session. (3) Coach sends a follow-up email within 24 h of session. The first two steps were coded from the session video recordings. For the follow-up emails, coaches uploaded copies of the emails to a secure online folder; fidelity to this step was confirmed by the presence of the email files. Treatment integrity as measured by the fidelity checklist and email review was 99% across all coaches.
Results
Sessions were scheduled for 20 min, but session length varied due to technology issues or the natural flow of activities. Average session length was 20 min, 22 s (range 16 min, 10 s–24 min, 8 s).
Paraprofessional Behavior
The results of the BIE coaching package on paraprofessional behavior are shown in Fig. 1, which displays both the percent of correctly implemented IT components in each session (line graph) and rate of IT trials presented by a paraprofessional in each session (bar graph).
Yara
Yara did not implement any IT trials during baseline. In her first intervention session, she delivered a rate of .15 trials/min, implementing 71% of IT components correctly. On day 2, Yara implemented only one IT trial (.07 trials/min) with 0% of components implemented correctly. On this day, Yara reported she felt sick. Additionally, there were over 8 min lost to technology issues during the session, resulting in only 14 min of codable time. The dyad subsequently moved to a different location in the classroom, which improved the internet connection. Yara gradually improved her implementation accuracy and met mastery criteria after 11 coaching sessions.
Brianna
Brianna’s rate of IT trials for the targeted self-advocacy phrase during baseline was zero. Her rate increased to .63 trials/min on the first day of intervention and averaged .59 trials/min (range .44–.69) over the remainder of the coaching sessions. Brianna implemented 90% of IT components correctly during the first day of coaching and averaged 94.8% (range 86–100) over the remaining coaching sessions. Brianna met mastery criteria after five coaching sessions. She continued delivering IT trials during generalization and maintenance sessions, averaging a rate of .43 trials/ min (range .35–.47). She also demonstrated 100% correctly implemented IT components across all generalization and maintenance sessions.
Gary
Gary did not implement any IT trials for the targeted self-advocacy statement during baseline. During the first day of coaching, Gary delivered IT trials at a rate of .62 trials/min and continued using IT trials at an increased rate throughout intervention, averaging .64 trials/min (range .55–.76). Gary implemented 98% of components correctly on the first day of coaching and 100% of the components correctly on day 2. Gary met criteria after 5 days of coaching. Gary’s rate of IT implementation remained high during his maintenance session (.54 trials/min with 100% of IT components delivered correctly), but he did not appear to generalize the skill, using only one IT trial across two generalization sessions for a rate of .04 trials/min.
Polly
Polly’s rate of IT trials for the targeted self-advocacy statement during baseline was zero. She implemented IT trials at a rate of .46 trials/min on the first day of coaching and averaged .42 trials/min across the remaining intervention sessions (range .31–.47). Polly averaged 97% correctly implemented components on her first day of intervention and stayed above 90% for the remainder of intervention. Polly met criteria after six sessions. Because of the end of the school year, data were not collected on maintenance and generalization for Polly.
Student Behavior
The results of the BIE coaching package on student behavior are shown in Fig. 2. The line graph represents the student’s rate of self-advocacy phrases, including prompted and unprompted phrases. The bar graph represents the rate of students’ independent use of self-advocacy phrases.
Yolanda
Yolanda did not use any of her self-advocacy target phrases during baseline, nor did she use them during the first 2 days of intervention. On the third day of intervention, her rate of usage increased to .27 utterances/min and the rate gradually increased throughout intervention. Her average rate across all intervention sessions was .26 utterances/min (range .14–.67). Yolanda first used a target phrase independently in Session 16. Her rate of prompted and unprompted use remained high during maintenance and generalization sessions (average: .57 utterances/min) and she used target phrases independently in both generalization sessions for an average rate of .09 utterances/min.
Bonnie
Bonnie did not use her target self-advocacy statement during baseline. Her usage immediately increased on the first day of intervention, where she used the phrase at a rate of .63 utterances/min. Bonnie continued to use the utterance at a high rate throughout intervention, averaging .55 utterances/min (range .44–.64) across intervention sessions. Bonnie increased her independent use of the phrase across intervention as well, utilizing the phrase independently at a rate of .10 and .15 utterances/min respectively in the last two intervention sessions. Bonnie continued to use the phrase at well above baseline levels during generalization and maintenance sessions, maintaining an average rate of .42 utterances/min during those sessions.
Gretta
Gretta did not use her self-advocacy target phrase during baseline. Her rate immediately increased to .61 utterances/min with the start of coaching and remained well above baseline levels for the remaining intervention sessions, averaging .63 utterances/min (range .55–.71). Gretta first used the target phrase independently in her fourth session of intervention. Gretta’s rate of target phrase use during generalization sessions was .035 utterances/min. She exhibited a high rate of independent use during maintenance (.32 utterances/min).
Pietra
Pietra did not use her self-advocacy target phrase during baseline. On the first day of intervention, Pietra’s rate increased to .46 utterances/min. She continued to use the phrase at an increased rate for the remaining intervention sessions, averaging .4 utterances/min across intervention sessions (range .26–.47). She first used her target phrase independently in her fourth intervention session for a rate of .06 utterances/min and increased her independent use of it in each of the remaining two intervention sessions (.11 and .21 utterances/min).
Tau-U Effect Sizes
Tau-U is an effect size estimate that can correct for trends present in single case data (Parker et al. 2011). It is considered preferable to other overlap statistics in single case design (e.g., Jamshidi et al. 2019). An online Tau-U calculator was used (Vannest et al. 2016) to calculate Tau-U for the rate of incidental teaching trials delivered per minute and for the percentage of IT trial components delivered correctly. Before calculating combined Tau-U statistics for each dependent variable, baseline conditions were analyzed for trend. There was no evidence of trend in any baseline condition across any dependent variable; therefore, no baseline corrections were made. Combined Tau-U statistics for each dependent variable were calculated. Combined Tau-U was 1.0 (95% CI [0.68, 1]; p < .00) for rate of IT trials per minute and 0.98 (95% CI [.67, 1]; p < .00) for percentage of IT trial components delivered correctly by the paraprofessionals. For student total target statements, combined Tau-U was 1.0 (95% CI [.68, 1]; p < .00). For student independent target statements, Tau-U was 0.36 (95% CI [0.05, 0.68], p = .02). With the exception of student independent target statements, which had a moderate effect size, all effect sizes were large.
Social Validity
All paraprofessionals filled out the anonymous online survey about the BIE intervention. Quantitative results can be seen in Table 2. Overall, paraprofessionals reported positive attitudes towards BIE coaching, improved ability to use incidental teaching, and improved confidence in their abilities. All agreed they would recommend BIE coaching to others. In open-ended comments about the BIE method, several of the paraprofessionals mentioned it could be hard at times to listen to both the coach and their student, but that it got easier over time. They also mentioned enjoying the immediate feedback; one reported, “I didn’t have to wonder if I was doing something correctly.” The survey suggested that paraprofessionals found their relationship with their coach to be positive. In open-ended questions, one paraprofessional expressed a desire for a more in-person aspect to the coaching process, particularly to get more familiar with the coach before “they are literally speaking in your ear.” Two paraprofessionals specifically mentioned the value of the email follow-up to the coaching sessions.
Discussion
This study investigated the effect of a BIE coaching package on paraprofessional use of incidental teaching to teach self-advocacy statements to four students with disabilities. With real-time coaching from a university-based coach, all four paraprofessionals successfully acquired this evidence-based practice. The four students also clearly benefited from incidental teaching. None of the students used their targeted self-advocacy statements in baseline, despite having self-advocacy goals for these statements on their current IEPs. After BIE coaching, all students practiced their target advocacy phrases on a regular basis and all used target phrases independently by the end of the intervention.
While all four paraprofessionals were successful in mastering incidental teaching, they differed somewhat in how quickly they mastered the skill. In particular, Yara showed a more gradual rate of acquisition compared to that of the other three paraprofessionals. A possible explanation for this is that Yara’s student, Yolanda, was the only student in the study with more than one acquisition target. This may have significantly increased the complexity of the intervention for Yara and contributed to a slower rate of acquisition. Future research should investigate whether paraprofessionals more easily master incidental teaching if only one target skill is identified during the initial acquisition phase.
This study contributes to the research on using BIE coaching in a school classroom, providing the first demonstration of BIE paraprofessional coaching from a remote location. Distance BIE coaching allowed university-based coaches to provide live coaching to staff in the classroom without being physically onsite, reducing travel time for the coaches and reducing time outside of the classroom for the paraprofessionals. While this early stage of distance BIE research utilized coaching by researchers, there is great promise for expanding BIE strategies into the paraprofessional support provided by special educators (Scheeler et al. 2018). Once research establishes the successful use of distance BIE supervision, the expertise of district specialists, supervisors, mentors, community partners, or higher education faculty can be leveraged efficiently without regard to geographical distance.
While incidental teaching is widely accepted as an evidence-based naturalistic language strategy in early childhood (Odom et al. 2010), this is one of few studies to investigate incidental teaching in K-12 classrooms. A particular strength of the current study is the range of students who benefitted from the intervention. The four students ranged from kindergarten to high school and varied greatly in communication repertoires, with one highly verbal student and another using an AAC device to communicate simple requests.
Of particular interest in this study was the performance of BIE technology in a school environment, particularly in a school building that was nearly a century old. Overall, technology issues were relatively few: out of a total of 555 min of coaching time, we experienced approximately 10 min of technology problems. There were two sources of technology problems: connection issues and SWIVL issues. Connection issues were minor and primarily limited to one classroom. Existing wireless routers in three of the four classrooms boosted internet capacity. For the room without a router, Yara’s classroom, there were several early BIE sessions with frequent connection problems. These issues were addressed in later sessions by having the dyad move to another part of the classroom. The other source of technical issues was the SWIVL robot, which occasionally unexpectedly swung the camera away from the dyad. The SWIVL robot was the most expensive item in the equipment used in this study, costing almost $500. We incorporated the SWIVL robot in this project based on experience in a previous study investigating the use of BIE coaching in a preschool classroom (Artman-Meeker et al. 2017). In that study, the teacher and the student moved around frequently and it proved difficult to keep the camera on the dyad. The purpose of the SWIVL robot is to have the camera follow the teacher if she and the student move around in the classroom. In this study of a K-12 environment, however, staying seated in one place during the coaching session did not present a problem for any of the dyads. Thus, the SWIVL robot did not end up being necessary and only contributed to potential technical issues. Thus, we would recommend that educators carefully consider the necessity of a SWIVL robot, or other similar technologies, before adding it to a BIE technology package.
Our experiences in this study also highlight the necessity for technology investment, assessment, and testing in real-world settings. Researchers on this project worked closely with school leadership to ensure that rooms had adequate technology infrastructure to sustain the BIE coaching intervention, but all costs for BIE equipment (aside from routers) was borne by the research team. Future research should attend to minimum technology requirements and the costs associated with each. Adequate time and personnel resources also must be considered. Our research team spent considerable time updating software, maintaining hardware, writing technology help guides, and securely backing up data. As school-based technology becomes more widespread and affordable, leaders can make investments to ensure schools are ready to support innovations like BIE coaching within their broader technology infrastructure. This is especially important in rural schools with limited access to specialized coaching expertise.
Limitations
A potential limitation of this study is that the package intervention prevents the assessment of the separate effects of either of the two components: didactic training and BIE coaching, in increasing the use of incidental teaching by the paraprofessionals. It is possible that didactic training influenced paraprofessional behavior without the addition of BIE coaching. However, the importance of coaching as a critical component of successful professional development has been well established in the teacher education literature (Joyce and Showers 2002; Kretlow and Bartholomew 2010; Scheeler et al. 2004), and thus, we felt justified in examining the effects of a package of didactic instruction followed immediately by coaching. Given the literature support for combining didactic instruction with coaching, it may be the appropriate time for the field to consider these two elements as one continuous process rather than as a package of discrete components.
While we included a treatment fidelity measure in our study, another potential limitation is that we collected limited procedural fidelity of the actual coaching that was conducted. We provided guidance on best practices delivering performance feedback using BIE for our coaches, but we did not measure their adherence to these practices. There is currently very little research on what BIE coaching should look like and how best practices in BIE coaching might differ from those of in-person performance feedback. Future research should examine what coaching behaviors contribute to the effective acquisition of new coachee skills when utilizing BIE technology and how BIE coaching implementation fidelity affects rates of coachee skill acquisition.
Finally, generalization is another potential limitation of the study. While all paraprofessionals acquired incidental teaching strategies, one paraprofessional, Gary, did not generalize the skill. We hypothesize that Gary did not have enough response examples during his training and coaching to prepare him to carry out incidental teaching in new environments. Gary, more than the other paraprofessionals, utilized an identical teaching activity from day to day and tended to use the same motivating situation for every trial. It is possible that Gary’s limited exposure to multiple exemplars of incidental teaching during the training and coaching phase led to difficulties in establishing generalized motivating situations for use of the target self-advocacy statement. Future research should examine the circumstances under which paraprofessionals generalize newly learned skills.
Directions for Future Research
This study opens several promising lines for future research. One area is the coach/coachee relationship during BIE coaching. The importance of a collaborative relationship in successful coaching has been established in the coaching literature (Barton 2018), but the methods of establishing this relationship in BIE coaching, particularly when the coach is remote, need further investigation. Strategies to mitigate the impact of technology issues on the coach/coachee relationship should be a particular focus. Technology problems may have played a part in Yara’s slower acquisition of incidental teaching relative to the other paraprofessionals, particularly because the technology problems were most frequent during the first few coaching sessions. Yara and her coach may have had a more fragmented and less cohesive coaching experience than the other dyads, preventing a strong coaching relationship from developing. Ways to establish a healthy partnership between the remote coach and coachee before intervention starts so that inevitable technology disruptions do not disrupt the coaching relationship should be investigated.
Finally, additional research is needed on aspects of implementation that would support sustained use of BIE coaching for paraprofessionals. The current study relied on university-based coaches, which may limit generalization. Future studies should address: (a) who would serve as BIE coaches, (b) what kind of training would be needed for these coaches, and (c) how technological issues would be addressed in classrooms without external support. In many settings, special educators are responsible for the day-to-day supervision and training of paraeducators. However, special educators report feeling poorly equipped for this role (Douglas et al. 2016). We argue that paraprofessionals deserve the same attention to professional development as all members of the educational team and should have access to skilled, sustained, job-embedded coaching. BIE coaching may be a particularly effective approach given its usability during contact time with students. Special educators may be uniquely poised to offer BIE coaching support since they know the students and their contexts well. Future research should examine the ways educators or systems take up an innovation like BIE coaching and the benefits and possible consequences associated with implementation.
Conclusion
Real-time coaching has been firmly established as an effective tool in helping educators learn how to implement evidence-based practices (Scheeler et al. 2004). The next step is to explore practical methods for providing this coaching to the paraprofessionals who implement daily instruction in schools. This study capitalized on emerging technology and demonstrated one method of coaching paraprofessionals in the classroom: distance coaching via bug-in-ear technology. This method holds promise in making real-time coaching a non-intrusive and sustainable method for developing paraprofessionals’ use of evidence-based practices regardless of geographical distance from coaching resources.
References
Artman-Meeker, K., Rosenberg, N., Badgett, N., Yang, X., & Penney, A. (2017). The effects of bug-in-ear coaching on pre-service behavior analysts’ use of functional communication training. Behavior Analysis in Practice,10(3), 228–241.
Barton, E. (2018). The use of email to coach early childhood teachers. Journal of Early Intervention,40(3), 212–228.
Brock, M. E., & Carter, E. W. (2013). A systematic review of paraprofessional-delivered educational practices to improve outcomes for students with intellectual and developmental disabilities. Research & Practice for Persons with Severe Disabilities,38(4), 211–221.
Carter, E., O’Rourke, L. O., Sisco, L. G., & Pelsue, D. (2009). Knowledge, responsibilities, and training needs of paraprofessionals in elementary and secondary schools. Remedial and Special Education,30(6), 344–359.
Charlop-Christy, M. H., & Carpenter, M. H. (2000). Modified incidental teaching sessions: A procedure for parents to increase spontaneous speech in their children with autism. Journal of Positive Behavior Interventions,2, 98–112.
Cornelius, K. E., & Nagro, S. A. (2014). Evaluating the evidence base of performance feedback in preservice special education teacher training. Teacher Education and Special Education,37, 133–146.
Douglas, S. N., Chapin, S. E., & Nolan, J. F. (2016). Special education teachers’ experiences supporting and supervising paraeducators: Implications for special and general education settings. Teacher Education and Special Education,39(1), 60–74.
Fallon, L. M., Collier-Meek, M. A., Maggin, D. M., Sanetti, L. M., & Johnson, A. H. (2015). Is performance feedback for educators an evidence-based practice? A systematic review and evaluation based on single-case research. Exceptional Children,81, 227–246.
Fenske, E. C., Krantz, P. J., & McClannahan, L. E. (2001). Incidental teaching: A not-discrete-trial teaching procedure. In C. Maurice, G. Green, & R. M. Foxx (Eds.), Making a difference: Behavioral intervention for autism (pp. 75–82). Austin, TX: PRO-ED.
Field, S., Martin, J., Miller, R., Ward, M., & Wehmeyer, M. (1998). Self-determination for persons with disabilities: A position statement of the Division on Career Development and Transition. Career Development for Exceptional Individuals,21, 113–128.
Garet, M. S., Porter, A. C., Desimone, L., Birman, B. F., & Yoon, K. S. (2001). What makes professional development effective? Results from a national sample of teachers. American Educational Research Journal,38(4), 915–945.
Giangreco, M. F., Broer, S. M., & Edelman, S. W. (2010). “That was then, this is now!” Paraprofessional supports for students with disabilities in general education classrooms. Exceptionality,10, 47–64.
Goodman, J. I., Brady, M. P., Duffy, M. L., Scott, J., & Pollard, N. E. (2008). The effects of “bug-in-ear” supervision on special education teachers’ delivery of learn units. Focus on Autism and Other Developmental Disabilities,23, 207–216.
Hart, B., & Risley, T. R. (1968). Establishing the use of descriptive adjectives in the spontaneous speech of disadvantaged preschool children. Journal of Applied Behavior Analysis,1, 109–120.
Hart, B., & Risley, T. R. (1975). Incidental teaching of language in the preschool. Journal of Applied Behavior Analysis,8(4), 411–420.
Hart, B., & Risley, T. R. (1982). How to use incidental teaching for elaborating language. Austin, TX: PRO-ED.
Hemmeter, M. L., Snyder, P., Kinder, K., & Artman, K. (2011). Impact of performance feedback delivered via electronic mail on preschool teachers’ use of descriptive praise. Early Childhood Research Quarterly,26, 96–109.
Individuals with Disabilities Education Act (IDEA), 20 U.S.C. § 1400 (2004).
Izzo, M., & Lamb, M. (2002). Self-determination and career development: Skills for successful transitions to postsecondary education and employment. A white paper for the Post-School Outcomes Network of the National Center on Secondary Education and Transition (NCSET) at the University of Hawaii at Manoa. Retrieved March 27, 2020 from http://www.ncset.hawaii.edu/publications/pdf/self_determination.pdf
Jamshidi, L., Heyvaert, M., Declercq, L., Fernandez-Castilla, B., Ferron, J. M., Moenaert, M., et al. (2019). Methodological quality of meta-analyses of single-case experimental studies. Research in Developmental Disabilities,79, 97–115.
Joyce, B. R., & Showers, B. (2002). Student achievement through staff development (3rd ed.). Alexandria, VA: Association for Supervision and Curriculum Development.
Kim, S., Koegel, R. L., & Koegel, L. K. (2017). Training paraprofessionals to target socialization in students with ASD: Fidelity of implementation and social validity. Journal of Positive Behavior Interventions,19(2), 102–114.
Kretlow, A. G., & Bartholomew, C. C. (2010). Using coaching to improve the fidelity of evidence-based practices: A review of studies. Teacher Education and Special Education,33(4), 279–299.
Kroeger, K. A., & Nelson, W. M. (2006). A language programme to increase the verbal production of a child dually diagnosed with down syndrome and autism. Journal of Intellectual Disability Research,50, 101–108.
Ledford, J. R., & Gast, D. L. (Eds.). (2018). Single case research methodology: Applications in special education and behavioral sciences (3rd ed.). New York: Routledge.
Ledford, J. R., Zimmerman, K. N., Chazin, K. T., Patel, N. M., Morales, V. M., & Bennett, B. P. (2017). Coaching paraprofessionals to promote engagement and social interactions during small group activities. Journal of Behavioral Education,26, 410–432.
Maggin, D. M., Fallon, L. M., Sanetti, L. M. H., & Ruberto, L. M. (2017). Training paraeducators to implement a group contingency protocol: Direct and collateral effects. Behavioral Disorders,38, 18–37.
McDonnell, J., & Jameson, J. M. (2014). What are the roles of paraprofessionals in inclusive schools? In J. McLeskey, N. L. Waldron, F. Spooner, & R. Algozzine (Eds.), Handbook of effective inclusive schools: Research and practice (pp. 94–106). Hoboken: Taylor and Francis.
McGee, G. G., Krantz, P. J., & McClannahan, L. E. (1985). The facilitative effects of incidental teaching on preposition use by autistic children. Journal of Applied Behavior Analysis,18, 17–31.
McGee, G. G., Morier, M. J., & Daly, T. (1999). An incidental teaching approach to early intervention for toddlers with autism. The Journal of the Association for Persons with Severe Handicaps,24(3), 133–146.
Miranda-Linne, F., & Melin, L. (1992). Acquisition, generalization, and spontaneous use of color adjectives: A comparison of incidental teaching and traditional discrete-trial procedures for children with autism. Research in Developmental Disabilities,13(3), 191–210.
Odom, S. L., Collet-Klingenberg, L., Rogers, S. J., & Hatton, D. D. (2010). Evidence-based practices in interventions for children and youth with autism spectrum disorders. Preventing School Failure,54, 275–282.
Ottley, J. R., & Hanline, M. F. (2014). Bug-in-ear coaching impacts on early childhood educators’ practices and associations with toddlers’ expressive communication. Journal of Early Intervention,36, 90–110.
Owens, T. L., Lo, Y., & Collins, B. C. (2019). Using tiered coaching and bug-in-ear technology to promote teacher implementation fidelity. The Journal of Special Education. https://doi.org/10.1177/0022466919852706.
Parker, R. I., Vannest, K. J., Davis, J. L., & Sauber, S. B. (2011). Combining nonoverlap and trend for single-case research: Tau-U. Behavior Therapy,42, 284–299. https://doi.org/10.1177/0145445511399147.
Powers, L. E., Geenen, S., Powers, J., Pommier-Satya, S., Turner, A., Dalton, L. D., et al. (2012). My life: Effects of a longitudinal, randomized study of self-determination enhancement on the transition outcomes of youth in foster care and special education. Children and Youth Services Review,34(11), 2179–2187.
Rock, M. L., Schumacker, R. E., Gregg, M., Howard, P. W., Gable, R. A., & Zigmond, N. (2014). How are they now? Longer term effects of eCoaching through online bug-in-ear technology. Teacher Education and Special Education,37, 161–181.
Scheeler, M. C., Morano, S., & Lee, D. L. (2018). Effects of immediate feedback using bug-in-ear with paraeducators working with students with autism. Teacher Education and Special Education,41(1), 24–38.
Scheeler, M. C., Ruhl, K. L., & McAfee, J. K. (2004). Providing performance feedback to teachers: A review. Teacher Education and Special Education,27, 396–407.
Schepis, M. M., Reid, D. H., Fitzgerald, J. R., Faw, G. D., van den Pol, R. A., & Welty, P. A. (1982). A program for increasing manual signing by autistic and profoundly retarded youth within the daily environment. Journal of Applied Behavior Analysis,15, 363–379.
Shogren, K. A., Wehmeyer, M. L., Palmer, S. B., Rifenbark, G. G., & Little, T. D. (2015). Relationships between self-determination and postschool outcomes for youth with disabilities. The Journal of Special Education,48(4), 256–267.
Test, D. W., Fowler, C. H., Wood, W. M., Brewer, D. M., & Eddy, S. (2005). A conceptual framework of self-advocacy for students with disabilities. Remedial and Special Education,26(1), 43–54.
U.S. Department of Education. (2016). 38th annual report to congress on the implementation of the Individuals with Disabilities Education Act. Retrieved June 28, 2019 from https://www2.ed.gov/about/reports/annual/osep/2016/parts-b-c/38th-arc-for-idea.pdf.
Vannest, K. J., Parker, R. I., Gonen, O., & Adiguzel, T. (2016). Single case research: Web based calculators for SCR analysis. (Version 2.0) [Web-based application]. College Station, TX: Texas A&M University. Retrieved Saturday 15th February 2020. Retrieved June 28, 2019 from www.singlecaseresearch.org.
Wacker, D. P., Lee, J. F., Padilla Dalmau, Y. C., Kopelman, T. G., Lindgren, S. D., Kuhle, J., et al. (2013). Conducting functional communication training via telehealth to reduce the problem behavior of young children with autism. Journal of Developmental and Physical Disabilities,25(1), 35–48.
Wehmeyer, M. L., & Palmer, S. B. (2003). Adult outcomes for students with cognitive disabilities three-years after high school: The impact of self-determination. Education and Training in Developmental Disabilities,38(2), 131–144.
Wong, C., Odom, S. L., Hume, K., Cox, A. W., Fettig, A., Kucharczyk, S., et al. (2014). Evidence-based practices for children, youth, and young adults with autism spectrum disorder. Chapel Hill: The University of North Carolina, Frank Porter Graham Child Development Institute, Autism Evidence-Based Practice Review Group.
Wood, W. M., Fowler, C. H., Uphold, N., & Test, D. W. (2005). A review of self-determination interventions with individuals with severe disabilities. Research and Practice for Persons with Severe Disabilities,30(3), 121–146.
Funding
This study received no funding.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare they have no conflict of interest.
Ethical Approval
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee (Team D Education; IRB ID: Study 00001094) and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards.
Informed Consent
Informed consent was obtained from all individual participants included in the study.
Statement on the Welfare of Animals
This article does not contain any studies with animals performed by any of the authors.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Rosenberg, N.E., Artman-Meeker, K., Kelly, E. et al. The Effects of a Bug-in-Ear Coaching Package on Implementation of Incidental Teaching by Paraprofessionals in a K-12 School. J Behav Educ 29, 409–432 (2020). https://doi.org/10.1007/s10864-020-09379-1
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10864-020-09379-1