The need for behavior analytic expertise in designing and implementing effective interventions has grown in recent years with demand for behavior analytic services more than doubling between 2012 and 2014 (Burning Glass Technologies, 2015). Increasingly, universities around the country offer coursework and supervised experience to train behavior analysts. These behavior analysts in training (henceforth referred to as pre-service behavior analysts) often participate in intensive supervised experiences providing behavioral services alongside certified behavior analysts. The contexts and quality of these experiences may vary, however. As the number of preparation programs increases, so has the need for techniques and knowledge about how best to prepare pre-service behavior analysts for practice in applied settings.

Preparing Behavior Analysts

Students training to become behavior analysts need a wide variety of skills, namely proficiency in the array of effective behavior analytic principles and strategies and the ability to train and coach others in those strategies. Typically, behavior analysts use indirect service models in which they train and coach a direct care provider such as a parent, a teacher, or a paraeducator to implement interventions (Bailey & Burch, 2010). Thus, it is important that applied behavior analysis preparation programs not only ensure that their graduates are well trained in behavior analytic principles and techniques but also that students experience effective methods for coaching others in how to implement those techniques. Receiving high-quality coaching from an experienced behavior analyst can promote students’ fidelity to behavioral strategies while introducing the practices they will eventually use to support clients’ use of behavioral approaches.

There is an extensive body of literature on promoting fidelity of implementation through effective coaching techniques. Side-by-side coaching, for example, has been used to ensure the fidelity with which individual behavior support plans are implemented in early childhood programs (e.g., Bethune & Wood, 2013) and the fidelity with which teachers implement class-wide positive behavior support strategies in early childhood settings (e.g., Stormont et al., 2015). Researchers have also examined the utility of behavioral consultation models on the implementation of individualized behavior support plans (e.g., Noell & Gansle, 2014); the plan implementation phase of behavioral consultation shares characteristics with common coaching models including observation, monitoring implementation, supporting plan revisions, and providing additional training as needed (Sheridan et al., 2013).

The coaching literature has highlighted the value of performance feedback on influencing professional practice (Scheeler et al., 2004), and performance feedback has been identified as an evidence-based practice for increasing educators’ treatment fidelity (Fallon et al., 2015). In particular, immediate feedback (i.e., feedback delivered in close temporal proximity to observed teaching or professional behavior) is an effective and efficient component of coaching. It can be contrasted with delayed feedback in which individuals receive feedback hours, days, or weeks after an observation. Immediate feedback provides an opportunity for praise or recognition of skillful practice (positive feedback) and timely correction of errors (constructive or corrective feedback). In a review of the literature, Scheeler et al. (2004) found that immediate feedback resulted in faster and more efficient acquisition of teachers’ learning targets than delayed feedback.

While desirable, the use of immediate feedback presents some challenges for coaching around challenging behavior. Being side by side with a teacher during an episode of a child’s behavior risks reactivity; children and adults will often act differently when they are aware of the observers’ presence and purpose (Cooper et al., 2007; McCambridge et al., 2014). In classrooms, feedback may sometimes be stigmatizing to the teacher, the child, or both. Ideally, the teacher, parent, or paraeductor would receive coaching or feedback in the moment without physical presence of the coach and without the child being able to hear the suggestions. Bug-in-ear (BIE) coaching may be ideal in these situations.

In BIE coaching, the direct service provider, such as a teacher or paraeducator, wears a wireless earpiece while delivering intervention, and a coach provides real-time feedback into the earpiece from a distance. For several decades, BIE coaching has been used to provide immediate feedback to individuals as they apply new skills (e.g., Crimmins et al., 1984). Early iterations of BIE coaching required that the coach and the coachee be present in the same room or clinic and connected through technology such as a two-way radio (Goodman et al., 2008), FM transmitter system (Scheeler et al., 2010), or cellular phones synced with wireless earpieces (McKinney & Vasquez, 2014; Ottley & Hanline, 2014). Technological innovations have expanded the possibilities for BIE coaching. An in-person classroom visit is no longer required to provide real-time feedback. For example, Rock et al. (2014) incorporated the use of Skype and wireless earpieces (i.e., Bluetooth) to provide BIE coaching remotely. In some instances, university supervisors were over 100 mi away from the individuals they coached. Observations took place by equipping teachers with laptop computers, webcams, and Bluetooth earpieces. Teachers and the university supervisor arranged times to observe via Skype, and the supervisor provided real-time feedback and suggestions during each observation. Rock et al. found long-term positive effects of BIE coaching on in-service teachers’ use of evidence-based practices and student responsiveness. Effects maintained 1–2 years after participating in BIE coaching.

There is now enough research supporting the use of BIE coaching in teacher education that it has been recommended as part of mentoring for new educators in rural areas (Israel et al., 2013). Given the similarity in the roles of teacher coaches and behavior analysts, BIE coaching may also be effective in preparing behavior analysts, both as a means to help them acquire behavior analytic techniques and also as a resource for coaching the direct care providers with whom they will be working. The ability to give feedback remotely may be particularly valuable, enabling university programs to coach behavior analysts who are acquiring supervised practical experience in remote locations and in giving practicing behavior analysts a tool for guiding teachers and parents in remote or rural areas.

Functional Communication Training and BIE Coaching

Functional communication training (FCT) is a behavior analytic technique that is designed to systematically teach socially acceptable alternatives to challenging behavior (Carr & Durand, 1985). Supported by an extensive literature base and considered an evidence-based practice in education, FCT is founded on identifying and teaching an appropriate behavior that serves the same communicative purpose as the challenging behavior (Carr, 1977). Functional communication training is typically preceded by a functional analysis (FA; Iwata et al., 1994) that identifies the function (i.e., the maintaining reinforcer) that the behavior is serving for the individual. To be most effective in decreasing challenging behavior, the replacement communication behavior taught during FCT must be more effective and efficient to access a reinforcer than the challenging behavior (Dunlap et al., 2015; Durand, 1990).

While FCT has been widely studied across educational and clinical settings and appears extremely effective in replacing challenging behaviors with more acceptable behaviors, it can be difficult to teach care providers to use. Skilled timing is required, so the appropriate child response is prompted (if necessary) after the child has encountered a behavioral trigger but before the inappropriate behavior has a chance to emerge. Because of the importance of timing in implementing FCT, immediate feedback for the implementer is essential. However, for all of the reasons discussed previously, having a coach side by side with the FCT implementer can be problematic. Thus, BIE coaching may be an ideal tool for providing immediate feedback during FCT implementation without being side by side with the coach.

While to our knowledge, there is no research evaluating the use of BIE coaching on FCT implementation in applied settings, a related technology called telehealth has been used to help families conduct FCT. In telehealth models, families or clients are coached through video conferencing technology. Often, families travel to locations close to their homes that are equipped with high-speed Internet, audio-recording and video-recording or streaming devices, and teleconferencing equipment. Wacker et al. (2013) used a fiber-optic communication network to coach parents to use FCT in regional outpatient clinics in Iowa. In a more recent study, parents received coaching on FCT via telehealth in their homes (Suess et al., 2014). Results suggested the parents were able to achieve a high degree of fidelity in their implementation of FCT and the children showed substantial reductions in challenging behavior.

While the use of remote technology to coach interventionists in FCT appears promising, there have not been studies directly investigating the utility of BIE coaching on FCT implementation in classrooms. BIE coaching may complement existing telehealth strategies because it offers a covert and private way to provide real-time feedback. Furthermore, there is very little research related specifically to strategies for developing the skills and competencies needed by pre-service behavior analysts. Behavior analysts must have a deep understanding of behavioral principles and be able to demonstrate these principles in their daily practice through common behavioral interventions such as FCT. The purpose of this study was to examine three research questions:

  1. 1.

    Is there a functional relation between BIE coaching and pre-service behavior analysts’ fidelity of implementation of FCT?

  2. 2.

    Do pre-service behavior analysts find BIE coaching socially valid and an acceptable means of receiving feedback?

  3. 3.

    Is there a functional relation between implementation fidelity of FCT and pre-school children’s challenging behavior and communication skills?

Method

Setting and Participants

This research was conducted in two special education classrooms within a university-affiliated early childhood center in the Pacific Northwest. The center offered a variety of programs including inclusive pre-school, kindergarten, and an intensive, ABA-based extended day program for children diagnosed with an autism spectrum disorder (ASD). This study took place in the extended day classrooms.

The children in this study were enrolled in an inclusive pre-school classroom for four half-days per week and in the extended day program for two half-days plus one full day per week. The extended day program was staffed with a ratio of approximately one educator to two children. Assistant teachers and other classroom staff consisted of graduate students in special education and volunteers. The lead teachers in the extended day class were all Board Certified Behavior Analysts® (BCBA®) as well as certified teachers.

Participants in this study included three pre-service behavior analysts and three pre-school-aged children. The three pre-service behavior analysts were enrolled in a graduate program in Applied Behavior Analysis and were completing their practicum in the extended day program previously described. To recruit these participants, the lead researcher visited a graduate class and described the study to students. Participants then volunteered to be in the study. Prior to participating in this study, all participants had completed graduate-level coursework in behavioral measurement, assessment, and behavioral intervention planning, including FCT.

To ensure an adequate base knowledge of FCT, interested participants were asked to complete an online assessment of FCT knowledge (Autism Internet Modules, 2015) and score at least 80%. All three interested participants completed the assessment successfully.

Three pre-school children were referred by the extended day classroom teachers. The lead researchers met with the extended day teachers, described the study, and asked the teachers to recommend children who might benefit from the FCT procedure. Teachers sent letters describing the study and consent forms home to families. Consent forms were returned for each of the three participating children. Each participant was then paired with one of the three children for the duration of the study. In all cases, the participant had spent time working as a classroom assistant with the child with whom she was paired prior to the start of the study. Participants were familiar with children’s instructional programs and goals.

Dyad 1: Beth and Carter. Beth and Carter were in an extended day classroom of eight children (six boys and two girls) and five full-time staff members. Beth was a 26-year-old Asian female with a bachelor’s degree in psychology. Before entering the graduate program, she had worked for approximately 4 years in a variety of paraprofessional roles in school, home, and clinical settings. She had received no additional training on FCT but had attended workshops on addressing challenging behavior through previous work sites.

Beth was paired with Carter, a 4-year-old boy diagnosed with ASD. Carter communicated using full sentences. Carter engaged in frequent disruptive behaviors, including crying, screaming, falling to the floor, elopement, and inappropriate verbal protest. His teacher reported that his behaviors were likely to occur across daily activities, including structured instructional time, meal times, free choice, and during transitions.

Dyad 2: Yanna and Zac. Yanna and Zac were in an extended day classroom of 16 children (13 boys and 3 girls) and 7 full-time staff members. Yanna was a 23-year-old Caucasian female with a bachelor’s degree in applied developmental psychology. She had worked for over 6 months as support staff in an autism classroom and a developmental group home. She had not received any training on FCT beyond her current graduate program.

For the study, Yanna was paired with Zac, a 4-year-old Caucasian boy diagnosed with ASD. Zac communicated using predominantly two to three word utterances. He engaged in challenging behaviors including loud vocalizations and falling to the floor. The behaviors were reported as most likely to occur during structured instructional periods and specifically during tasks that were reported to be difficult for the student.

Dyad 3: Bailey and Aaron. Bailey and Aaron were in the same classroom as Yanna and Zac. Bailey was a 30-year-old Hispanic female with a bachelor’s degree in special education and early childhood education/child development. She had been an early childhood special education teacher for 6 years prior to entering the graduate program. She had not received any training on FCT beyond her current graduate program.

Bailey was paired with Aaron, a 5-year-old Caucasian boy diagnosed with ASD. Aaron communicated primarily through gestures, inconsistently using two to three word utterances. His challenging behaviors included screaming, crying, whining, and self-injurious behavior in the form of face slapping and scratching. Disruptive behavior was reported to occur throughout the day but was most likely in unfamiliar situations and following the removal of highly preferred activities.

Coaches. Both coaches were advanced doctoral students in Special Education and were also BCBAs. One of the coaches led the participants’ weekly practicum seminar and had 2 years of experience supervising pre-service behavior analysts. The second coach had several years of experience coaching families. Both coaches were involved in the design of this study’s coaching protocol and met bimonthly with the research team prior to and during the study to discuss coaching implementation and procedures.

Dependent Measures and Materials

Each session was approximately 20 min and consisted of semi-structured play with a child’s individual social goals embedded in the play. Typical materials involved toy animals, houses, cars, car ramps, train tracks, play dough, sequencing picture cards, and other play materials. In most cases, the adult participant was working with the target child of this study and one other student, who was not in the study, during the session.

Each adult participant wore a Bose© Bluetooth Head Set Series 2 (BIE device) during each baseline and intervention session. The BIE device was connected to a wifi-enabled IPod Touch on a tripod stand positioned 1 m from the dyad. A research assistant ensured that the BIE device was connected to the IPod, positioned the IPod to record the dyad, and initiated a FaceTime™ call with the coach (or researcher during baseline). The coach received the call on a MacBook Air laptop computer connected to the wireless Internet in a university office. All calls were recorded using Ecamm Call Recorder for FaceTime. To address connectivity issues, a new wireless Internet router was installed in the children’s classroom prior to beginning the study. All FaceTime calls were completed on an authenticated university wireless network; student information was protected through FaceTime’s peer-to-peer connections and end-to-end encryption. Videos were collected and stored in compliance with federal laws (Family Educational Rights and Privacy Act, Health Insurance Portability and Accountability Act). Furthermore, all participants (or child participants’ parents) signed consent and release forms authorizing the use of FaceTime and the use of video for educational purposes.

Sessions began when the participant placed the BIE device and indicated she was ready to begin. Observations ended after the coach had observed at least 20 min of semi-structured play time or if the situation called for the session to be terminated (e.g., semi-structured play time ended). Times in which audio or video feed was frozen were excluded from calculating the length of sessions. Average time lost to audio or video failure for Beth was 28 s (range 0–180 s), for Bailey was 36 s (range 0–180 s), and for Yanna was 32 s (range 0–213 s). Excluding these times lost to technology failure, the average length of sessions was 18.85 min (range 11–24 min) for Beth, 20.43 min for Bailey (range 18–28 min), and 20.25 min for Yanna (range 17.45–25 min). All data were coded from video recordings.

Participant Behavior

Data were collected on the rate of FCT trials provided by each adult participant (trials per min) and the fidelity with which the trials were implemented. To measure fidelity, a seven-step FCT protocol was developed by the research team (Table 1). Trials began when the participant intentionally established a pre-determined motivating operation (e.g., “Play my way”). For each trial, each step of the FCT protocol was coded as “yes,” “no,” or “not applicable.” Fidelity was calculated by adding the number of steps coded yes across trials, dividing by the total number of steps possible across trials (excluding steps that were not applicable), and multiplying by 100.

Table 1 Functional communication training protocol
Full size table

Child Communication and Challenging Behavior

The primary measure for each child was the rate of target communication identified in the FCT plan. Appropriate target behaviors for each child were determined from the results of a brief functional analysis (BFA; described below). For all three children, the target behavior was determined to be a request for “One more minute.” Each trial was coded for independent or prompted use of the target communication. The rate of child target communication was calculated by dividing the frequency of independent or prompted use of the target behavior in each session by the duration of the session (in minutes).

Challenging behavior was operationally defined and measured for each child. Carter’s challenging behavior was defined as shouting, shrieking, or screaming at a louder than typical volume; growling; pulling an object towards him; or pushing others. Shouting within the context of play (i.e., taking on a role) or talking loudly without agitation was not coded. Zac’s challenging behavior was defined as yelling, crying, or whining that lasted longer than 3 s. Aaron’s challenging behavior was defined as crying, yelling, face poking, face slapping, and head banging. Repetitive behaviors or behaviors that were appropriate parts of pretend play were not coded. A 10-s partial interval system was used to code challenging behavior. If challenging behavior occurred at any point in the 10-s interval, it was coded as yes. To summarize percentage of intervals with challenging behavior, we calculated yes / (yes + no) and multiplied by 100.

Research Design

A concurrent multiple baseline design across participants was used (Baer et al., 1968). That is, the onset of the intervention was staggered across time for the three dyads. Phase changes were made based on visual analysis of participants’ fidelity to FCT procedures. The first and second authors, both doctoral-level BCBAs, evaluated the graphs for changes in level, trend, and variability after each treatment session. Phase change decisions were made based on changes in participants’ data patterns in treated tiers concurrent with stability in non-treated tiers.

Procedures

Functional Assessment

Multiple sources of evidence were collected to identify a hypothesized function for each child’s challenging behavior. First, a member of the research team interviewed each child’s special education teacher using the Functional Assessment Interview Form (FAI; O’Neill et al., 1997). This instrument helped the research team operationally define the target behaviors and gain a clear description of the contexts in which behavior was most likely to occur. Next, each teacher was asked to complete the Motivation Assessment Scale (MAS; Durand, 1990) to specify situations in which the child was likely to use the operationally defined challenging behavior. On the MAS, teachers were asked to rate 16 items about the behavior on a scale of 0 (never) to 6 (always). Responses to the 16 items were then sorted into four categories representing possible functions of the behavior: sensory, escape, attention, and tangible. Finally, a member of the research team observed the child from an observation booth attached to the child’s extended day classroom. The observation was used to confirm the presence of the operationally defined challenging behavior and to record the contexts in which it was observed to occur. The results of the FAI, MAS, and direct observation were used to inform the subsequent BFA and the design of the FCT plan.

Brief Functional Analysis

A BFA (Northup et al., 1991) was conducted for each child participant. Participants were exposed to a series of 5-min assessment sessions that each included a single condition (e.g., demand, attention, tangible, and control/free play). Not all participants experienced all conditions; the conditions for each child were determined after indirect functional assessment and subsequent direct observation in the classroom.

Two members of the research team and the child were in the testing room during the assessments. Each BFA continued until the researchers agreed on the function of behavior. An average of 12.3 sessions was conducted across participants (range 7–15 sessions), and all BFAs were completed within 2.5 h. All sessions of each BFA were video recorded, and the occurrence of challenging behavior was recorded from these videos using 10-s partial interval recording. See Fig. 1 for the results of the BFA. For Carter, results suggested that target behaviors were maintained by tangible reinforcement, and it appeared that access to routines was a specific characteristic of the tangible function. For Aaron, challenging behavior was determined to serve a tangible function. While elevated levels of behavior were observed in the control and escape conditions, further analysis using the video recordings indicated that in these two conditions, target behaviors continued in the reinforcement interval, suggesting that Aaron was not receiving the controlling reinforcer for the behaviors. Finally, Zac’s BFA included few sessions in each condition due to the clarity of the results indicating a tangible function and because of concerns around Zac’s signs of potential distress during the BFA sessions. The researchers used the results of each BFA to develop each child’s FCT plan.

Fig. 1
figure 1

Percentage of intervals with challenging behavior during brief functional analyses for Carter, Zac, and Aaron

Full size image

Baseline

Adult participants were given the written FCT plan designed by the lead researchers. Each participant had one 30-min meeting with the lead researchers to discuss the plan and study procedures and to ask questions prior to implementing the plan. To reduce reactivity to the study procedures, participants were provided with the BIE devices and recording equipment during this training session. The participants practiced using the equipment with the lead researcher and participated in role play and guided practice using the equipment. One participant (Beth) requested an additional meeting with her cooperating teacher to discuss a plan for engaging a non-target child during the study. This request was honored, and the lead researcher attended the meeting. No additional training was provided during this meeting. Participants then began baseline sessions. During baseline sessions, participants were asked to implement the FCT procedures outlined in the plan and were asked to wear the BIE device, but no coaching was provided. A research assistant initiated a FaceTime™ call from the classroom to the research office, and a research assistant recorded the call.

Coaching

During the coaching phase, coaches provided immediate feedback to the participants via the BIE device. Coaches were expected to (a) provide positive feedback to the participant for successful implementation of FCT, (b) make suggestions for improved implementation of FCT based on the FCT protocol, (c) ensure an adequate number of FCT trials, and (d) provide an e-mail summary of feedback after the session. To ensure an adequate number of FCT trials, if a minute had elapsed since the last FCT trial, the coach would look for an appropriate pause in the participant/child interaction to suggest a FCT trial (Appendix Figure 4). Feedback consisted of statements that were supportive (praise) and constructive (prompts or suggestions). Supportive comments consisted of praise such as “great job delivering your prompt!” or “Way to respond immediately by removing the demand.” Constructive prompts or suggestions included statements such as “Be ready with your prompt.” Feedback was kept short to convey information quickly. Coaches responded to participant questions and concerns during the coaching session, and coaches sent a follow-up e-mail within 24 h of the observation. In the e-mail, the coach thanked the participant for the session, summarized any conversations, answered participants’ questions, and offered a reminder of the next visit. The e-mail did not contain any new or additional feedback. The coaching phase continued until participants achieved 90% fidelity over three consecutive days or until the end of the school year.

Maintenance

Maintenance data were collected for Beth and Yanna. The school year ended before maintenance data could be collected for Bailey. During maintenance, participants conducted FCT but did not wear the BIE device and received no feedback. Maintenance data were collected eight calendar days after intervention ceased for Beth and five calendar days after intervention ceased for Yanna.

Procedural Fidelity

A trained research assistant scored each video-recorded session and assessed implementation fidelity (Billingsley et al., 1980) of BIE coaching. On average, coaches provided positive feedback on 81% of trials (range 53–96%). Coaches prompted less frequently, with an average of 0.74 prompts per session (range 0–3). In keeping with the protocol to allow 1 min to lapse before looking for the next opportunity to prompt a trial, coaches were considered to demonstrate fidelity to the coaching protocol if they prompted a participant within 2.5 min of a previous trial and an opportunity was present. Coaches did so 100% of opportunities. On average, coaches made 2.63 suggestions per session (range 0–11). Coaches spent, on average, approximately 2 min (range 9–373 s) per session providing feedback or suggestions to participants in real time. Follow-up e-mails were provided after 100% of Yanna’s sessions, 83% of Bailey’s sessions, and 44% of Beth’s sessions.

Interobserver Agreement

A graduate research assistant naïve to the study design coded 40% of randomly selected sessions across all participants, dependent variables, and data collection phases. Interobserver agreement (IOA) was calculated on participant behavior (number of FCT trials and fidelity of implementation) as well as child target communication and challenging behavior. IOA was also calculated on child challenging behavior during each participant’s BFA.

IOA for a number of FCT trials was calculated using a total agreement approach (Kennedy, 2005) by summing the total number of trials for each observer per session and dividing the smaller number by the larger number and multiplying by 100. IOA was 100% for number of trials across all participants and all conditions. For fidelity of FCT implementation, each response of the seven-step FCT protocol was scored as an agreement or disagreement; an agreement was scored if both observers recorded the response as occurring or not occurring in a trial. IOA was calculated by dividing the total number of agreements by the sum of agreements plus disagreements and multiplying by 100. Across all participants and research conditions, average IOA for FCT fidelity was 87.81% (range 50–98.7%). IOA data by participant and experimental condition are presented in Table 2. The low level of agreement (50%) in one session was due to a baseline session in which a participant only provided two trials.

Table 2 Interobserver agreement by participant and experimental condition
Full size table

IOA for child target communication was calculated by dividing the number of agreements on the occurrence or non-occurrence of prompted or unprompted child communication by the total number of opportunities for child communication (i.e., trials) and multiplying by 100. Average IOA for child communication was 87.45% (range 50–100%) and is presented by participant and experimental condition in Table 2. The low level of agreement (50%) in one session was due to a baseline session in which a participant only provided two trials, so there were only two opportunities for child target communication.

IOA on child challenging behavior was calculated using interval agreement during study procedures and each child’s BFA (Kennedy, 2005). Each interval was coded as an agreement or disagreement. Agreement was scored when both observers coded the same response (occurrence or non-occurrence of challenging behavior) for that interval. The total number of agreements was divided by the sum of agreements plus disagreements and multiplied by 100. Average IOA for child challenging behavior during study procedures was 97.61% (range 91.67–100%) and is presented by participant and experimental condition in Table 2. Average IOA for the BFA was 99.65% (range 97–100%). Average IOA was 100% for free play/control conditions, 100% for attention conditions, 99.62% (range 97–100%) for tangible conditions, 100% for demand conditions, and 97% (no range) for tangible routines conditions.

Social Validity

Social validity describes the extent to which the consumer (i.e., participant, caregivers) is satisfied with the effects, experiences, and importance of an intervention (Wolf, 1978). At the conclusion of the study, participants anonymously completed a six-item satisfaction survey. Each item was rated on a scale from 1 (strongly disagree) to 4 (strongly agree). Participants were asked to respond to three related open-ended question (i.e., “Why or why not?”). At the end of the survey, there was one additional open-ended question asking participants to compare their experience receiving BIE coaching to traditional forms of coaching. All participants completed the survey and returned it via campus mail.

Results

Participant Behavior

All participants’ FCT fidelity is shown in Fig. 2. Figure 2 also shows the rate of FCT trials delivered per session. Beth (top panel, Fig. 2) demonstrated relatively high and stable FCT fidelity during baseline (that is, after having been given only a written plan). After being stable for two sessions, Beth’s FCT fidelity showed a slight upward trend to 78.57% on the third baseline session. With BIE coaching, Beth immediately increased to 91.96% fidelity and remained above baseline levels throughout intervention (M = 95.02%; range 84.96–99.37%). She reached a maximum of 99.37% fidelity during BIE coaching session 8. There were no overlapping data points between baseline and BIE coaching. Beth also increased the rate of FCT trials she delivered during the sessions. Beth’s rate immediately increased from 0.33 trials per minute in her last baseline session to 0.9 trials per minute during her first BIE coaching session. With the exception of one session, Beth’s rate of FCT trials remained above baseline levels (M = 1.14 trials per minute; range 0.55–1.91 trials per minute). Beth maintained her skills during the maintenance probe, continuing to implement FCT with 85% fidelity while initiating 0.95 trials per minute during the session.

Fig. 2
figure 2

Participants’ percentage fidelity to the FCT protocol and rate of FCT trials per minute. The break in Bailey’s abscissa indicates an absence of 2 weeks

Full size image

Yanna (middle panel, Fig. 2) showed low FCT fidelity during baseline, completing only an average of 38.57% of the steps correct during baseline trials. She also had a very low rate of FCT trials, averaging only 0.14 FCT trials per minute during baseline (range 0.09–0.22 trials per minute). Yanna’s implementation fidelity immediately increased to 72% with BIE coaching and improved to 96.2% in the second session, remaining above 90% for the remainder of the sessions (M = 93.49%; range 72.02–96.1%). There were no overlapping data points between baseline and BIE coaching. Yanna’s rate of trials also increased significantly with the onset of coaching: increasing from 0.14 average during baseline (range 0.09–0.22 trials per minute) to an average of 0.93 trials per minute during intervention (range 0.55–1.2 trials per minute) with no overlapping data points between baseline and BIE coaching. Yanna maintained her skills during the maintenance probe, demonstrating 96.10% fidelity to the FCT plan while implementing 0.55 trials per minute during the maintenance session.

Bailey’s FCT fidelity (bottom panel, Fig. 2) was low during baseline. During session 2, Bailey offered no FCT trials. Following session 2, Bailey’s target child was absent for 2 weeks (indicated by a break in the data path). Upon the child’s return, Bailey’s FCT fidelity remained stable between 50 and 60%. Bailey offered a very small number of FCT trials during baseline sessions: Bailey had an average rate of only 0.18 FCT trials per minute during baseline sessions (range 0–0.4 trials per minute). Bailey’s rate of trials and implementation fidelity increased immediately with BIE coaching. Her fidelity of implementation climbed to 82.3% during the first BIE coaching session and remained stable during the coaching phase. Her rate of trials increased immediately to 1.06 trials per minute on the first day of coaching and remained high, averaging a rate of 0.93 trials per minute during the intervention phase (range 0.5–1.36 trials per minute). There were no overlapping data points between baseline and BIE coaching for Bailey. Because of the end of the school year, a maintenance probe was not conducted for Bailey.

Child Behavior

The results of the FCT intervention on child behavior are shown in Fig. 3, which displays each child’s use of the target communication in rate per minute as well as the percentage of intervals per session with challenging behaviors.

Fig. 3
figure 3

Percentage of intervals with challenging behavior and rate of target communication by Carter, Zac, and Aaron during 20-min sessions. The break in Aaron’s abscissa indicates an absence of 2 weeks

Full size image

Carter (top panel, Fig. 3) showed a steady decreasing trend in challenging behavior during baseline. He emitted challenging behavior, on average, during only 3.6% of intervals (range 0–6.59%) and had no challenging behavior during his final baseline session. Carter’s rate of challenging behavior increased in variability during the BIE coaching intervention, and three of the nine data points exceeded baseline levels. He averaged challenging behavior in 5.29% of intervals (range 1.67–10%). Carter engaged in challenging behavior during 10% of the intervals during the maintenance probe.

Carter demonstrated independent use of the target communication at an average rate of 0.48 communications per minute during baseline (range 0.05–0.85), but his target communication also showed a clear downward trend in baseline. During the final baseline sessionn, he used target communication independently only once. With the introduction of BIE coaching on his FCT plan, Carter’s independent communication increased in level and showed an increasing trend. Although data were variable, five of the nine intervention sessions exceeded baseline levels. He increased his rate of independent use of the target communication to 0.93 communications per minute during BIE coaching (range 0.35–1.51) and demonstrated a rate of 0.89 independent communications per minute during the maintenance probe.

Zac (middle panel, Fig. 3) displayed almost no challenging behaviors during baseline or intervention, averaging 0% intervals during baseline and averaging 0.2% intervals (range 0–0.94%) during the BIE coaching phase. Zac’s independent use of the target communication was low during baseline and showed a slight downward trend (M = 0.14 per minute; range 0.05–0.25). His communication increased in level immediately to 1.18 per minute during the first intervention session and remained above baseline levels until intervention session 7. Following a downward trend in independent communication, Zac’s final three intervention sessions were near baseline levels. Across the BIE coaching phase, he averaged 0.69 communications per minute (range 0.24–1.18). It is notable that Zac spontaneously began using an alternate communication strategy (“No thanks”) that was not coded in child communication, although it was reinforced as functional communication. Zac used the target communication at a rate of 0.30 communications per minute during his maintenance probe.

During baseline, Aaron (bottom panel, Fig. 3) exhibited variability in challenging behaviors (M = 5.43% of intervals; range 0–12.5%). His behaviors spiked on his first day back after an extended absence (session 3; 12.5% intervals). His challenging behaviors increased in level and variability during the BIE coaching phase, where he displayed challenging behavior in an average of 15.31% of the intervals (range 1.67–29.25%). No maintenance data were collected for Aaron.

Aaron’s independent use of the target communication was low and stable during baseline (M = 0.04 per minute; range 0–0.20 per minute). His independent rate of communication immediately increased to 0.84 communications per minute on the first day of BIE coaching and averaged 0.63 per minute over the BIE coaching phase (range 0.13–1.12 per minute). Because of the end of the school year, a maintenance probe was not conducted for Aaron.

Social Validity

Table 3 shows the results from the social validity questionnaire. All participants returned the questionnaire. The participants were positive about the intervention, agreeing that BIE coaching improved their ability to use FCT and that they would recommend it to others. While they found BIE coaching to be somewhat distracting, they said that it was less disruptive than other forms of coaching. In their comments, participants suggested combining BIE coaching with in-person debriefing to increase collaboration with the coach. Two participants also mentioned difficulty working with more than one student during BIE coaching.

Table 3 Social validity
Full size table

Discussion

This study examined the effects of BIE coaching on pre-service behavior analysts’ implementation of FCT. BIE coaching was associated with increases in the rate of FCT trials each participant initiated and in the fidelity with which participants implemented FCT. This finding means that adults created more intentional opportunities for children to communicate, and adults provided more systematic instruction around those opportunities (i.e., prompt hierarchy, reinforcement, data collection). As a result, each child increased his rate of independent communication relative to baseline.

This is the first study to use BIE coaching to promote individualized, function-based communication during authentic play experiences in a school setting. The previous FCT literature using BIE coaching has primarily focused on families and has been implemented in clinical settings or homes (Wacker et al., 2013; Suess et al., 2014). Although other studies have used BIE coaching to increase teachers’ use of communication strategies (e.g., Ottley & Hanline, 2014), the strategies were not explicitly linked to individual children’s functional communication needs. Previous research with school-aged populations has focused on math and literacy instruction using large group instruction (Rock et al., 2014) or discrete trial teaching (Scheeler et al., 2004).

Finally, this study is one of the first to examine the training and remote/BIE supervision of pre-service behavior analysts. As the demand for behavior analytic services grows (Burning Glass Technologies, 2015), the preparation and job-embedded learning of these professionals will be of critical importance. This study provided pre-service behavior analysts with high-quality coaching and increased their competence and fluency at implementing behavioral practices. Thus, our procedures provide a model for how pre-service behavior analysts could support their own clients in the future.

There was no clear functional relation between FCT and decreases in child challenging behavior in this study. This was an interesting finding given the long history of effectiveness in studies of FCT (e.g., Carr & Durand, 1985). There are several possible explanations for this outcome. First, incorrect functions of behavior may have been identified. The BFA data strengthen confidence in hypothesized functions, but it is possible that more salient contingencies may have been in place for each child. Next, unlike many FCT studies, the children in this study had fairly low levels of challenging behavior during baseline. This likely had to do with the specific context in which this study took place: a highly structured setting with a strong orientation to the principles of behavior and a high level of behavioral supports already in place. For example, participants used a number of visual supports and used individualized token economy systems.

Second, the FCT protocol we taught required the creation of frequent natural opportunities for using the target communication. The participants were coached to frequently (approximately once per minute) evoke circumstances that, for the children, had been associated with challenging behavior. For example, the children were frequently asked to give up a preferred toy or had a preferred play routine interrupted. It is possible the increased frequency of such demands caused challenging behaviors to rise. In addition, the quantity and quality of antecedent interventions in place in the classrooms may have prevented children from being exposed to relevant antecedents outside of the study. Experiencing the antecedents may have increased the children’s levels of challenging behavior.

It is also possible that the rate of FCT opportunities that were presented in this study was simply too high. We coached participants to maximize the amount of practice students received during the 20-min FCT session, but in more natural conditions, it may be appropriate to arrange the opportunities out over a longer period of time. This adjustment may result in fewer challenging behaviors for the child. However, based on our own experience coaching pre-service professionals around FCT, we suspect that increasing opportunities for practice may be critical for children’s skill generalization and maintenance. Very few FCT studies have looked at the issue of maintenance of the replacement behaviors taught (e.g., Carr and Durand, 1985; Wacker et al., 2013). The child has typically had many opportunities, often over years, to practice the challenging behavior, and they will need intensive opportunities to practice the new behavior if it is going to successfully replace the challenging behavior over time and settings. Thus, if coaching time is limited, it may be worth the possibility of a temporary increase in challenging behaviors, if it provides intensive, prolonged practice of the replacement behavior in natural settings.

The issue of child challenging behavior has implications for the acceptability and feasibility of coaching intervention such as BIE. FCT has long been recognized as an effective practice for increasing communication and decreasing challenging behavior (Carr & Durand, 1985), but it does require time for adults to learn to use it with fidelity. During this learning period, adults may not use FCT as it is intended and they may not see the results they desire. If coachees perceive FCT as ineffective because the child’s challenging behavior appears to be worsening, it is possible that they will abandon the intervention in the coach’s absence. This potential outcome has implications for the duration, dosage, and frequency of coaching needed to influence long-term behavior change. It may be important either to distribute coaching on the FCT trials so that challenging behavior does not increase or for coaching to continue until the presumably short-term increase in challenging behaviors subsides. Coaches must be thoughtful about matching their services to coachees’ needs.

Limitations and Directions for Future Research

While this study has many promising implications for preparing future behavior analysts, there are limitations as well. First, the project experienced intermittent technology failures. Despite installing a wireless high-speed Internet router in the classrooms, we still experienced occasional poor video quality, garbled audio, or frozen video. The amount of time lost from technical difficulties during coaching averaged 47.04 s per session across participants (range 0–213 s). However, technology issues were generally resolved, and coaching resumed within a minute or two. Although technology failure was accounted for in data analysis, this limitation likely mimics real challenges practitioners and other researchers face when using Internet technology for distance coaching and supervision.

Second, the generalizability of this study may be limited by the presence of a research staff member in each classroom during all sessions. The staff member initiated the FaceTime™ calls and provided technology support as needed. The researcher did not interact with participants or provide any feedback, but this layer of technological support limits the generalizability of the study’s findings to schools without this resource.

The timing of the academic school year also impacted the current study. The study took place between April and June, so the school year ended before optimal maintenance data could be collected. Ideally, coaches would have provided feedback as participants learned to thin the schedule of reinforcement within the FCT protocol, but this was impossible in the present study.

Finally, the coaching protocol used in this study offered coaches only broad guidelines for their coaching (e.g., ensure trials approximately every minute, provide positive feedback) (Appendix Table 4). Coaching was seen as individualized to each participant’s implementation of her child’s FCT plan, and coaches were encouraged to work within the protocol to provide meaningful feedback. Coaches were provided with the FCT protocol for each child and a decision chart for prompting and reinforcing participant behavior. Therefore, it was difficult to summarize the fidelity with which coaches provided feedback in a simple percentage or metric. Rather, we videotaped coaching sessions and monitored the amount and frequency of participant and coach talk during sessions. This is consistent with other reports of coaching fidelity in the early childhood coaching literature (Artman-Meeker et al., 2015). The depth with which coaching was monitored in the current study is consistent with other reports in the literature on web-mediated coaching interventions (i.e., Powell et al., 2010). Future research should articulate the specific steps and essential elements of BIE coaching. Researchers should investigate the dosage of positive feedback, suggestions, or prompts that is associated with increased fidelity to interventions. This is an important step in defining BIE coaching and making recommendations for practice.

Conclusion

There is very little research related specifically to strategies for developing the skills and competencies needed by pre-service behavior analysts. Behavior analysts must have a deep understanding of behavioral principles and be able to demonstrate these principles in their daily practice. They must be able to fluently implement common behavioral strategies such as FCT. This study presents valuable evidence of the effectiveness of BIE coaching on pre-service behavior analysts’ use of FCT strategies and the effects on child communication.