Telehealth is defined as “the use of electronic information and telecommunication technologies to support long-distance clinical health care, patient and professional health-related education, public health, and health administration” (American Telemedicine Association, 2017). An emerging body of literature has evaluated the use of telehealth to provide applied behavior analytic (ABA) services to individuals with autism spectrum disorder (ASD; Ferguson et al., 2019). A majority of the ABA procedures delivered via telehealth have been implemented in home-based settings by caregivers, whereas a researcher trained and supervised the caregiver in implementing the procedures (Ferguson et al., 2019).

Telehealth is an acceptable service delivery mechanism for ABA interventions and has led to positive outcomes for decreasing problem behavior and increasing skill acquisition in individuals with ASD (Ferguson et al., 2019; Unholz-Bowden et al., 2020). In addition, telehealth reduces the cost associated with ABA services (Ferguson et al., 2019; Horn et al., 2016; Lindgren et al., 2016); is an effective platform for parent training (e.g., increasing implementation skills; Meadan & Daczewitz, 2015); and behavioral skills training (BST) delivered via telehealth is as effective as BST in person for teachers, therapists, and parents implementing behavioral interventions with individuals with ASD (Neely et al., 2016).

Literature regarding best practices for providing behavior analytic services via telehealth has also increased (e.g., Lee et al., 2015; Lerman et al., 2020; Pollard et al., 2017; Rios et al., 2018). Lee et al. (2015) provided recommendations for setting up telehealth services and troubleshooting possible technology problems. Two recent studies (Lerman et al., 2020; Rios et al., 2018) reported challenges and potential solutions that practitioners may experience when providing behavior analytic services via telehealth. Finally, Pollard et al. (2017) describe ethical considerations when developing and providing services via telehealth.

Though some practitioners or organizationsFootnote 1 may provide remote supervision, limited literature exists on training, best practices, implementation guidelines, and troubleshooting resources when providing supervision to supervisees via telehealth. Indeed, behavioral interventions are most effective when professionals implementing those interventions are adequately supervised (Shapiro & Kazemi, 2017). However, limited empirical research on supervision may affect the quality of these experiences. A recently published systematic review of supervision research further suggests the need for empirical studies to determine if the tools and resources provided in the existing body of literature produce effective outcomes (Kranak et al., 2023).

Within the field of behavior analysis, supervision is defined as “improving and maintaining the behavior analytic, professional, and ethical repertoires of the supervisee and facilitating the delivery of high-quality behavior analytic services to the supervisee’s clients” (Behavior Analyst Certification Board [BACB], 2018). Individuals who provide supervision under the umbrella of this definition (i.e., board certified behavior analyst [BCBA], board certified assistant behavior analyst [BCaBA]) must be credentialed through the BACB and are commonly referred to as supervisors. A supervisor oversees individuals providing behavior analytic services and those accruing fieldwork hours to become certified (BACB, n.d.b). The supervised individual is called the supervisee (e.g., registered behavior technician, BCaBA, trainee). A supervisee is “any individual whose behavioral service delivery is overseen by a behavior analyst within the context of a defined, agreed upon relationship” (BACB, 2020). The BACB requires that all supervisees receive supervision when providing behavior analytic services.

Supervision of ABA services is critical because it can increase the quality of behavior analytic services (LeBlanc & Luiselli, 2016), positively affecting treatment outcomes (Kranak et al., 2023). In addition, supervision can promote professional development (e.g., establish professional values and increase interpersonal skills) of the supervisor and supervisee and can help the field of ABA by developing future practitioners that have the appropriate competencies to create successful and socially significant behavior change (Brodhead et al., 2018; LeBlanc et al., 2012; LeBlanc & Luiselli, 2016; Sellers et al., 2016a; Turner et al., 2016). Supervision can also increase the likelihood of ethical employee behavior and result in greater consumer protection (Brodhead & Higbee, 2012). Finally, supervisor support and high-quality supervision may help reduce supervisee burnout and job dissatisfaction while increasing job productivity (Dounavi et al., 2019; Jimenez-Gomez et al., 2021; Plantiveau et al., 2018).

Supervision Barriers

Supervision does not occur without barriers, regardless of the modality in which supervision is provided (e.g., face-to-face, telehealth). We define supervision barriers as something that hinders the supervision of the supervisee and the quality of services provided to the supervisee’s clients. An example of a supervision barrier is when the supervisor (hereafter referred to as the BCBA) does not devote ample time to the supervisee (e.g., the BCBA only has 15 min to meet with the supervisee when the situation demands 30 min of supervision), and as a result, does not provide an appropriate amount of feedback. In another example, the supervisee could struggle with interpersonal skills (e.g., is rude), making it difficult for them to receive and then subsequently implement feedback (Sellers et al., 2016b). When a barrier occurs during supervision (e.g., BCBA does not provide feedback), that barrier may affect the organization where the BCBA and supervisee work because the staff may be dissatisfied with the supervision provided and, as a result, leave the organization (DiGennaro Reed & Henley, 2015; Sellers et al., 2016b). In addition, if a barrier occurs during supervision (e.g., lack of access to materials), that barrier may put the clients at risk if the barrier affects providing high-quality behavior analytic services (Sellers et al., 2016b).

In 2019, Sellers and colleagues surveyed BCBAs to gather information about supervision practices and to identify any barriers respondents might experience while providing supervision. Though Sellers et al. provided information on barriers BCBAs face when supervising trainees, it is unclear what modality the BCBAs provided the supervision in (e.g., face-to-face supervision, supervision provided via telehealth). As a result, it is unknown if any of the respondents provided supervision via telehealth and if barriers would be similar or different based on the modality. Further research is warranted because one should not assume that the barriers experienced in one context (e.g., face-to-face) would be the same barriers experienced in another context (e.g., telehealth) without supporting data. Likewise, though it may seem intuitive that telehealth may occasion barriers that differ from face-to-face supervision, such assumptions also require caution without supporting data.

Without having information about barriers encountered when providing supervision via telehealth, researchers are less likely to study ways to mitigate them, which negatively affects the body of research that informs applied practice. Therefore, it is of benefit to the field of ABA to understand the barriers that may occur under different supervisory contexts, including telehealth. Second, knowing the barriers and potential strategies used to address the barriers can provide BCBAs with practical tools to address or mitigate the barriers if the onset of barriers cannot be prevented.

In addition, the COVID-19 pandemic brought tremendous challenges to the field of ABA. The rapid transmission of COVID-19 and stay-at-home orders demanded an urgent transition from face-to-face to telehealth service delivery to maintain continuity of care (Cox et al., 2020). Though the COVID-19 pandemic was a rare disaster in terms of scale, it represents an important context to study rapid changes in service delivery as recipients of behavioral interventions are certainly not immune to future disruptions in service delivery (e.g., natural disasters or political upheaval) that may demand an urgent transition in service modality (White et al., 2023). Therefore, understanding barriers in remote supervision experienced before the COVID-19 pandemic, as well as during, may further inform considerations for practice and future research.

The information mentioned above will likely provide critical guidance to inform future scholarship on the supervision of behavioral interventions via telehealth, which in turn should increase the quality of telehealth supervision BCBAs provide, resulting in a higher quality of care and greater consumer protection (Brodhead & Higbee, 2012; Sellers et al., 2019). Therefore, we conducted a survey study to ask the following research questions: (1) What are the barriers BCBAs experience when providing supervision to other BCBAs or individuals pursuing a BCBA credential (i.e., trainee) who are providing behavioral services via telehealth to individuals with ASD in the United States? (2) What strategies do BCBAs use to address or mitigate the barriers that arise during supervision of behavioral services via telehealth to individuals with ASD in the United States? In addition, we (3) sought to compare the challenges BCBAs experienced in providing supervision via telehealth before and during the COVID-19 pandemic to inform recommendations for how a BCBA may quickly transition service delivery modalities (i.e., from in-person to telehealth) in the future. Finally, our survey and results are presented in an open-science repository to inspire replication of our work and greater transparency in instrument and data-sharing practices to inform and inform future more direct empirical research in the supervision of behavioral interventions.

Method

Participants

Participants were recruited through the BACB mass email service, which is an email contact list of all registered certificants, in May 2021. To use the BACB mass email service, the researchers were required to pay a fee for the initial email and the 1-week reminder email to be sent to potential participants. Participants were recruited using voluntary sampling, which consisted of explicitly calling for volunteers (Remler & Van Ryzin, 2011) through the BACB mass email service. All potential participants resided within the United States and had a BCBA credential or a BCBA credential with a doctoral designation (BCBA-D). According to the BACB, a maximum of 42,405 individuals potentially qualified to receive the email invitation. However, the BACB indicated that individuals could independently opt out from receiving emails from the BACB at any time; therefore, the number of individuals who received the survey was lower and fluctuated over time (i.e., initial email was sent to 18,983 individuals, 1-week reminder email was sent to 19,154 individuals).

Potential participants were provided with a research participation and consent form at the beginning of the survey. Following the form, participants were asked to select one of two options: (1) I have read the informed consent and agree to participate, or (2) I have read the informed consent and do not agree to participate. If potential participants indicated they read the informed consent and agreed to participate, they were then screened for the following inclusion criteria: (1) if the potential participant held a BCBA credential or BCBA credential with a doctoral designation (BCBA-D) in good standing (i.e., active certification status); (2) if the potential participant currently, or in the past 6 months, provided supervision to another BCBA or individual pursuing a BCBA credential; (3) if the potential participant currently or in the past 6 months, provided supervision via telehealth to another BCBA or individual pursuing a BCBA credential; and (4) if the potential participant provided supervision to someone that provided behavior analytic services to individuals with ASD. Participants had to consent to participate and meet all four inclusion criteria to be included in the study. Finally, it is important to note that the researchers opted to include any BCBAs who provided supervision via telehealth (i.e., BCBAs who provided supervision via telehealth to another BCBA or individual pursuing a BCBA credential) to collect data on all areas of remote supervision rather than focusing only on BCBAs who provided supervision to an individual pursuing a BCBA or registered behavior technician credential.

Materials

The final survey was created on Qualtrics and included 34 multiple-choice, side-by-side, rank, and fill-in-the-blank questions that were broken up across four sections: (1) inclusion criteria; (2) supervision load and supervision meeting logistics; (3) experience when providing supervision via telehealth; and (4) demographic information (see Supplementary MaterialsFootnote 2). Four survey questions consisted of initial survey screening questions to determine if the participants met the four inclusion criteria mentioned above. Six questions were asked about the participant’s supervision load (e.g., number of individuals the BCBA was supervising via telehealth) and supervision meeting logistics (e.g., frequency of telehealth supervision meetings, length of meetings). Twelve questions were asked about the participant’s experiences when providing supervision via telehealth (e.g., training received, supervisee barriers experienced, supervisor barriers experienced, strategies used to address or mitigate supervisee and supervisor barriers experienced). Finally, 12 questions asked for demographic information (e.g., age, number of years worked as a BCBA, current organization setting).

For the questions that included a time frame (i.e., currently or within the past 6 months), the time frame was selected because the researchers were interested in learning about the experiences of supervisors who were currently or recently providing supervision via telehealth. Currently or within the past 6 months provided the researchers with that specific information. For the questions regarding supervision and telehealth, definitions of each were displayed on the screen. Telehealth was defined as “the use of electronic information and telecommunication technologies to support long-distance clinical health care, patient and professional health-related education, public health, and health administration” (American Telemedicine Association, 2017). Supervision was defined as “improving and maintaining the behavior-analytic, professional, and ethical repertoires of the supervisee and facilitating the delivery of high-quality behavior analytic services to the supervisee’s clients” (BACB, 2018). In addition, for the questions regarding supervision and telehealth, participants who worked at multiple organizations were asked to think about the organization they primarily worked in when answering. For the questions regarding barriers experienced, the barriers were broken into two areas: supervisee barriers (19 options) and supervisor barriers (25 options). Finally, for the questions regarding strategies to address or mitigate the barriers, strategies were broken into two areas: supervisee barrier strategies used (24 options) and supervisor barrier strategies used (18 options).

Amazon.com gift cards for $10 were used as incentives for participants to complete the survey. After completing the survey, participants interested in entering a drawing to receive financial compensation for their time were directed to email their contact information to an email address affiliated with the university that the researchers did not have access to. The 33 participants that entered their names into the drawing received an Amazon.com gift card.

Procedure

Expert and Content Reviews

The primary researcher created the survey. Expert and content reviews of the survey were then completed. The expert reviews were completed by two doctoral-level professionals who held a BCBA credential, currently or previously served as a faculty member at a research institution, had substantial applied experience, and well-established scholarly records in either supervision or telehealth. The researchers emailed the survey to the expert reviewers and asked them to review it and provide feedback. The expert reviews resulted in changes in the wording of six questions within the survey to improve question clarity. In addition, one question format was changed from a multiple-choice question format to a side-by-side question format to gain information regarding how often (i.e., never, rarely, usually, always, not applicable) individuals were typically present during supervision telehealth meetings. Finally, one question was added to ask if the participants had received training on how to provide supervision in the past 6 months instead of only asking if the participants had received training on how to provide supervision via telehealth in the past 6 months. The content reviews were sent to and completed by five BCBAs who provided supervision via telehealth to another BCBA or an individual pursuing a BCBA credential. The content reviews resulted in one change to the survey: adding a nonappplicable choice for the question regarding how often individuals were typically present during supervision telehealth meetings.

Survey Distribution

The initial survey email was sent to potential participants directly through the BACB mass email service. Potential participants had access to the survey for 2 weeks after the initial survey email was distributed. The survey was open for 2 weeks due to the nature of the initial email and the 1-week reminder email set up through the BACB mass email service, providing participants equal time to complete the survey. Researchers wanted to ensure that potential participants had the same time to complete the survey (i.e., 1 week) following each email. Each potential participant could access the survey only once using the link provided in the email to prevent individuals from submitting multiple responses. One week after the initial email was sent to potential participants, a reminder email identical to the initial email was sent to potential participants. The link to the survey was the same in both emails to ensure that once an individual clicked on the link and completed the survey, they could not submit another response. Recruitment of participants and data collection were conducted from May 4, 2021, until May 18, 2021.

Data Analysis

Following the survey distribution, participant data were analyzed using descriptive statistics, specifically focusing on frequencies, percentages, and measures of central tendency in ways consistent with similar studies (e.g., Hajiaghamohseni et al., 2020; Sellers et al., 2019). In particular, descriptive statistics were used for respondent demographics, supervision load and supervision meeting logistics, barriers experienced (first research question), and strategies used to address or mitigate barriers that arose (second research question).

Results

According to the BACB, 42,405 individuals qualified to potentially receive the email invitation. The metrics from the BACB that were provided after the completion of data collection indicated that the initial email was sent to 18,983 individuals. Of those individuals, 2,115 people opened the email, and 121 people clicked the survey link included in the email. The reminder email was sent to 19,154 individuals (this number was higher because the number of certificants who subscribe to the listserv is fluid). Of those individuals, 2,246 people opened the email, and 151 people clicked the survey link included in the email (for a total of 272 people).

Of the 272 people who opened the survey link, a total of 150 responses were collected: 23 participants (15.3%) did not complete the entire survey, two participants (1.3%) did not currently hold a BCBA credential in good standing, 42 participants (28.0%) did not provide supervision in the past 6 months, 10 participants (6.7%) did not provide supervision via telehealth in the past 6 months, and four participants (2.7%) did not supervise individuals who provided behavior analytic services to individuals with ASD. Therefore, 81 of the original 150 responses were discarded because they did not meet our initial a priori inclusion criteria requirements. Sixty-nine participants (46.0%) completed the entire survey, consented to participate, and met all four inclusion criteria, and therefore were included in the data analyses. The survey response rate was 0.8% (150 out of 19,154 individuals).

Respondent Demographics

See Supplementary Materials for tables and specific demographic information for the 69 participants included in our analysis. The mean age of the participants was 36.8 years (range: 25–65), the median age was 35 years, and the mode was 36 years (n = 7, 10.1%). A majority of the participants were female (n = 60, 87.0%), were white (n = 57, 82.6%), and held a master’s degree (n = 55, 79.7%). A total of 25 states were represented, with the most participants working in California (n = 14, 20.3%) and Michigan (n = 10, 14.5%). When comparing the demographic information of the 69 participants to the BACB data of certificants (BACB, n.d.a), it was found that the demographics from the present study were reflective of the demographics of the profession at that time.

When participants were asked how many years they have worked as a BCBA, the most frequently selected answers were 6 or more years (n = 29, 42.0%) and 2 years (n = 11, 15.9%). When participants were asked how many years they have worked at their current organization, the most frequently selected answers were 6 or more years (n = 17, 24.6%), less than 1 year (n = 11, 15.9%), and 1 year (n = 11, 15.9%). When asked how many employees worked at their current organization, the most frequently selected answers were 1–25 employees (n = 19, 27.5%) and 251 or more employees (n = 15, 21.7%). When participants were asked what their current organization setting was, the most frequently selected answers were multiple settings (e.g., ABA agency clinic-based, ABA agency home-based, school [n = 23, 33.3%]), and ABA agency home-based (n = 20, 29.0%).

Supervision Load and Supervision Meeting Logistics

Most participants indicated that they had been providing supervision via telehealth over the past 6 months (n = 48, 69.6%). In addition, a majority of participants indicated they were supervising one (n = 18, 26.1%), two (n = 16, 23.2%), or three (n = 15, 21.7%) individuals. When participants were asked to define the population of individuals with ASD receiving telehealth services, the most frequently selected answers were elementary school (n = 18, 26.1%) and early intervention (n = 8, 11.6%). Additional populations included high school and above (n = 5, 7.2%); multiple populations consisting of early intervention, preschool, and elementary school (n = 5, 7.2%); and multiple populations consisting of early intervention, preschool, elementary school, and junior high (n = 5, 7.2%). When asked, on average, how frequently supervision meetings occurred, a majority of the participants indicated that they held meetings one time per week (n = 42, 60.9%). The remaining participants indicated they held meetings two times per week (n = 19, 27.5%), three times per week (n = 2, 2.9%), and four or more times per week (n = 1, 1.4%). Five participants (7.2%) indicated they would prefer not to answer how often they held meetings. When asked about the length of the supervision meetings, most participants indicated they were typically 40–60 min (n = 28, 40.6%) or 61–75 min long (n = 17, 24.6%). The remaining participants indicated the length of their meetings were 76–90 min (n = 8, 11.6%), 31–45 min (n = 7, 10.1%), 91+ min (n = 5, 7.2%), or 16–30 min long (n = 4, 5.8%).

Experiences When Providing Supervision Via Telehealth

When participants were asked if they had received training on providing supervision in the past 6 months, 33 participants (47.8%) indicated that they had. Thirty-two participants (46.4%) indicated that they had not received training on supervision in the past 6 months but had received training before. Four participants (5.8%) indicated that they had not received training in the past 6 months or before. When participants were asked if they had received training on providing supervision via telehealth in the past 6 months, 26 participants (37.7%) indicated that they had. Twenty-eight participants (40.6%) indicated that they had not received supervision via telehealth in the past 6 months but had received training before. Fifteen participants (21.7%) indicated that they had not received training in the past 6 months or before.

Barriers Experienced

When asked which supervisee barriers participants experienced, the most frequent was internet connectivity issues (n = 46, 66.7%), and the second most frequent was distractions during the supervision meeting (n = 25, 36.2%; see Fig. 1). A majority of participants experienced one (n = 10, 14.5%), two (n = 14, 20.3%), three (n = 12, 17.4%), or four (n = 10, 14.5%) supervisee barriers, with a mean of 3.3 barriers (see Fig. 2). Six participants (8.7%) indicated they had not experienced any supervisee barriers.

Fig. 1
figure 1

Percentage of each supervisee barrier experienced across participants

Full size image
Fig. 2
figure 2

Number of supervisee versus supervisor barriers experienced across participants. Note. For number of barriers, we only listed the number of barriers participants indicated that they experienced. If a number was not listed, zero participants indicated that was the number of barriers they experienced

Full size image

When asked which supervisor barriers participants experienced, the most frequent supervisor barrier that occurred was the ability to model or demonstrate strategies (n = 29, 42.0%), and the second most frequent supervisor barrier was obstruction of view or supervisee out-of-lens view (n = 28, 40.6%; see Fig. 3). Most participants experienced one (n = 12, 17.4%) or two (n = 16, 23.2%) supervisor barriers, with a mean of 2.9. Nine participants (13.0%) indicated they had not experienced any supervisor barriers.

Fig. 3
figure 3

Percentage of each supervisor barrier experienced across participants

Full size image

Strategies Used to Address or Mitigate Barriers Experienced

Overall, the most frequently used supervisee strategies to address or mitigate each barrier varied between (1) setting clear expectations for the supervisee; (2) clarifying expectations for the supervisee; (3) supervisor providing training on technology and video conferencing software to the supervisee; and (4) the supervisee gaining access to internet services (see Tables 5 and 6 in the Supplementary Materials for strategies used to address or mitigate each supervisee barrier and for the most frequent strategy used).

Overall, the most frequently used supervisor strategies to address or mitigate each barrier varied between (1) the supervisor not using a strategy to address or mitigate the barrier that arose; (2) setting clear expectations for themselves; and (3) developing and implementing a self-management strategy for themselves (see Tables 5 and 6 in the Supplementary Materials for strategies used to address or mitigate each supervisee barrier and for the most frequent strategy used).

COVID-19 Pandemic

When participants were asked if they had provided telehealth supervision before the COVID-19 pandemic (i.e., before March 2020), 22 participants from the original sample (31.9%) indicated they had. Of those 22 participants, 20 (90.9%) indicated they would continue to provide supervision via telehealth after the COVID-19 pandemic.

Forty-seven participants from the original sample (68.1%) indicated they had not provided telehealth supervision before the COVID-19 pandemic. Of those 47 participants, 39 (83.0%) indicated they would continue to provide supervision via telehealth after the COVID-19 pandemic. The remainder of the results will be analyzed to compare participants who conducted telehealth supervision before the pandemic and those who had not.

Barriers Experienced

Participants Who Provided Supervision via Telehealth Prior to COVID-19 Pandemic

For participants who provided supervision via telehealth prior to the COVID-19 pandemic (n = 22), the most frequent supervisee barrier that occurred was internet connectivity issues (n = 11, 50.0%), and the second most frequent supervisee barrier was distractions during the supervision meeting (n = 8, 36.4%; see Figure 1 in Supplementary Materials). A majority of participants experienced two (n = 5, 22.7%), four (n = 4, 18.2%), or six (n = 4, 18.2%) supervisee barriers, with a mean of 3.05 barriers (see Figure 2 in Supplementary Materials). Three participants (13.6%) indicated they had not experienced any supervisee barriers.

When asked which supervisor barriers participants experienced, the most frequent supervisor barriers that occurred were distractions during supervision meetings (n = 7, 31.8%), time constraints (n = 7, 31.8%), and internet connectivity issues (n = 7, 31.8%; see Figure 3 in Supplementary Materials). A majority of participants experienced one (n = 4, 18.2%) or two (n = 4, 18.2%) supervisor barriers, with a mean of 2.9 barriers (see Figure 2 in Supplementary Materials). Four participants (18.2%) indicated they had not experienced any supervisor barriers.

Participants Who Did Not Provide Supervision via Telehealth Prior to COVID-19 Pandemic

For participants who indicated they had not provided supervision via telehealth prior to the COVID-19 pandemic (n = 47), the most frequent supervisee barrier that occurred was internet connectivity issues (n = 35, 74.5%), and the second most frequent supervisee barrier was distractions during the supervision meeting (n = 17, 36.2%; see Supplementary Figure 4). A majority of participants experienced one (n = 7, 14.9%), two (n = 9, 19.1%), three (n = 10, 21.3%), or four (n = 6, 12.8%) supervisee barriers, with a mean of 3.45 barriers (see Supplementary Figure 5). Three participants (6.4%) indicated they had not experienced any supervisee barriers.

When asked which supervisor barriers participants experienced, the most frequent supervisor barrier that occurred was the ability to model or demonstrate strategies (n = 23, 48.9%), and the second most frequent supervisor barrier was obstruction of view or supervisee out-of-lens view (n = 22, 46.8%; see Supplementary Figure 6). A majority of participants experienced one (n = 8, 17.0%) or two (n = 12, 25.5%) supervisor barriers, with a mean of 2.94 barriers (see Supplementary Figure 5). Five participants (10.6%) indicated they had not experienced any supervisor barriers.

Strategies Used to Address or Mitigate Barriers Experienced

Participants Who Provided Supervision via Telehealth Prior to COVID-19 Pandemic

Overall, the most frequently used supervisee strategies to address or mitigate each barrier varied between (1) clarifying expectations for the supervisee; (2) the supervisee gaining access to internet services; (3) setting clear expectations for the supervisee; and (4) providing immediate feedback to the supervisee.

Overall, the most frequently used supervisor strategies to address or mitigate each barrier varied between (1) developing and implementing a self-management strategy for themselves; (2) clarifying expectations for themselves; (3) setting clear expectations for themselves; (4) completing an evaluation on supervision activities; (5) completing an evaluation on the supervisee’s performance; (6) contacting a friend in the field of behavior analysis; and (7) reading journal articles on the issue.

Participants Who Did Not Provide Supervision via Telehealth Prior to COVID-19 Pandemic

The most frequently used supervisee strategies to address or mitigate each barrier varied between (1) setting clear expectations for the supervisee; (2) the supervisee gaining access to a computer, tablet, or smartphone with a webcam and microphone; and (3) the supervisor providing training on technology and video conferencing software to the supervisee.

The most frequently used supervisor strategies to address or mitigate each barrier varied between (1) setting clear expectations for themselves; (2) developing and implementing a self-management strategy for themselves; and (3) the supervisor had not used a strategy to address or mitigate the barrier that arose.

Discussion

The purpose of the present study was to identify barriers BCBAs experienced while providing supervision via telehealth and to identify strategies BCBAs used to address or mitigate those barriers. Within the telehealth supervision context, we found the most common barriers were internet connectivity issues (supervisee barrier) and the ability to model or demonstrate strategies to the supervisee (supervisor barrier). The latter represents a potentially serious concern regarding effective training and subsequent implementation of behavioral interventions because ineffective instructional strategies, on behalf of the supervisor, may result in poor instructor fidelity and treatment outcomes. This finding underscores the importance of continual evaluation of instructional strategies and feedback mechanisms provided via telehealth (see Sipila-Thomas et al., 2022, for an example). We do not doubt the efficacy of behavioral interventions and positive reinforcement; however, our survey results suggest that changes in the environment produce unique contextual considerations for training and treatment implementation. Though it is possible that what BCBAs report does not align with what occurs (e.g., procedural fidelity of treatment may remain high even if a BCBA reports difficulty in modeling that treatment in a remote context), we urge caution in making such an assumption without direct evaluation. Regardless, these results highlight that direct empirical research on supervision via telehealth, especially barrier mitigation, is more important than ever.

Additional barriers identified by our survey respondents included distractions encountered during the supervision meeting (supervisee barrier), scheduling conflicts (supervisee barrier), and obstruction of view or supervisee out-of-lens view (supervisor barrier). Sellers et al. (2019) found that the most common barrier was a lack of time to adequately prepare for supervision meetings and develop a tracking system to monitor the skills and knowledge the supervisee has mastered. Additional barriers reported by Sellers et al. consisted of the cost of materials, lack of access to resources, lack of access to examples, the uncertainty of supervision requirements, and uncertainty about how to teach and measure specific skills of the supervisee. A key finding of this study is that barriers experienced in one context (e.g., supervision via telehealth) may not be the same barriers experienced in another context (e.g., face-to-face supervision). Though this finding may seem intuitive, this study is the first to provide preliminary empirical support that BCBAs report they experience unique supervisory challenges in telehealth settings. As a result, training on providing supervision should be specifically tailored to the context in which it is provided (e.g., telehealth or face-to-face).

A limitation of our study is that we did not survey barriers reported in both face-to-face and telehealth contexts. We recommend a study comparing barriers encountered face-to-face to and telehealth contexts as an avenue for future research, allowing researchers to better compare reported similarities and differences because responses would represent the same sample of participants. This comparison would also help inform the creation and empirical evaluation of supervisory training programs for face-to-face and telehealth supervision contexts.

Due to the timing of our survey, we had the unique opportunity to compare the nature of challenges BCBAs experienced in providing supervision via telehealth before and during the COVID-19 pandemic to inform recommendations for how a BCBA may quickly transition service delivery modalities (i.e., from in-person to telehealth) in the future. To our knowledge, this is the first study to conduct such an analysis. Our preliminary results indicated that for respondents with experience providing telehealth supervision before the COVID-19 pandemic, the two most frequent supervisee and supervisor barriers were internet connectivity issues and distractions that arose during supervision meetings. Respondents with no experience providing telehealth supervision before the COVID-19 pandemic reported similar barriers with respect to their supervisees. Therefore, internet connectivity and environmental distractions are probable barriers a BCBA may encounter when providing supervision via telehealth, regardless of the social, political, or environmental context.

However, supervisors with no prior experience with supervision via telehealth before the COVID-19 pandemic indicated the two most common barriers they experienced as BCBAs were (1) the ability to model or demonstrate strategies and (2) issues regarding the obstruction of view or supervisee being out-of-lens view. Above, we described the importance of addressing barriers to modeling and demonstrating behavioral strategies and again echo those statements. Moreover, given that obstruction of view of the supervisee was a common barrier experienced by BCBAs with no prior experience providing supervision via telehealth before the COVID-19 pandemic (as well as our general sample of respondents), it is time to consider the role of advanced technology (e.g., telepresence robots) that could actively work and move to mitigate visual obstructions. Though this technology comes at a higher cost, our findings indicate that its direct evaluation in future research is worth exploring.

Our initial findings suggested that previous training did not appear to prevent barriers from occurring. We conducted a post-hoc analysis to evaluate if there was a relationship between the number of years a participant was a BCBA and the number of barriers they experienced. A linear regression established that the number of years a participant was a BCBA did not result in differences in the number of supervisee or supervisor barriers they experienced (see Supplementary Materials). This finding is important to note as it underscores the importance that supervisor resources and training should not focus solely on newly credentialed or unexperienced BCBAs and instead should focus on all BCBAs, regardless of the number of years they have been a BCBA or the level of experience they have (Lerman et al., 2020).

Seventy-eight percent of participants reported they had received training on providing supervision via telehealth in the past 6 months or before. This number may initially appear high; however, in previous research, Hajiaghamohseni et al. (2020) found that 99.1% of their participants had prior supervision training. As an alternative, it is alarming that 21.7% of participants reported not receiving training on providing telehealth supervision in the past 6 months or before. This finding is concerning because regardless of the level of expertise a BCBA has when providing supervision or services in person, it is important not to assume that supervision skills will transfer to a telehealth context without specific training (Lerman et al., 2020). Poor training practices may also become professional habits and negatively affect treatment outcomes for the clients (Sellers et al., 2016a). It is worth reminding that the present survey was administered in the context of a public health emergency (i.e., COVID-19). Therefore, focused training on supervision via telehealth may not have been possible at that time and, in some cases, ethically justifiable (see Cox et al., 2020). Future research could conduct a follow-up survey to determine if the present results were partly due to the context or if there is an issue with training more broadly. Regardless, future research may explore the development and subsequent evaluation of component skills necessary to conduct supervision via telehealth so those skills may be established in graduate training programs to better equip the next generation of BCBAs.

For participants who reported having at least some training in providing supervision via telehealth, the type of training varied across participants. For example, participants may have contacted a friend in the field of behavior analysis, engaged in self-guided learning, observed another professional, or participated in a conference. However, the dosage, quality of training, and when training was received is unknown. The present study did not ask these specific questions. Future research could gather specific information about the amount of training received, when the training was received, and the quality of training received. In addition, future research could directly evaluate the amount of training, and the type of training supervisors need to prevent barriers from occurring and identify strategies that can be used to address the barriers that occur in a timely manner. This information could then inform employers and university training programs on how much training supervisors should receive to ensure they provide effective supervision.

Recommendations for Organizations and Individual BCBAs

Below, we summarize recommendations for organizations and BCBAs to mitigate barriers when providing telehealth supervision. These recommendations are based on the preliminary outcomes of our study and the collective expertise of both authors. We have provided an expanded list of recommendations in Table 1, which includes accompanying citations to provide the reader with a starting point for further inquiry. A primary intention of Table 1 is to serve as a “stand-alone” tool for organizational leaders and BCBAs to inform telehealth supervision processes. We encourage interested researchers to empirically evaluate our recommendations and for readers to think critically about them and consider their organizational or environmental needs before adopting or modifying them. For readers interested in how to use behavioral systems analysis to improve supervisory processes, see Brodhead (2020), Brodhead et al. (2022), and Brodhead and Oteto (2023). We also recommend the Performance Diagnostic Checklist-Human Services (Carr et al., 2013) as a starting point for identifying interventions to improve personnel performance.

Table 1 Recommendations to address and/or mitigate supervisee and supervisor barriers
Full size table

Organization Level

At the organizational level, a starting point for barrier reduction and prevention is for an organization to train all BCBAs on how to provide supervision via telehealth before beginning the supervision process. When creating this training, the organization should ensure they are using the process of evidence-based practices, which consists of using the best available research (e.g., effective training methods), considering the values of the client, the context (e.g., supervision via telehealth), and expertise of the individual providing the training to identify the practices to use (Brodhead et al., 2018; Slocum et al., 2014). In addition, the organization should tailor the training to meet the needs of providing supervision via telehealth. As our results show, the unique context of telehealth demands considerations different from those in a face-to-face context.

However, it may not always be possible to eliminate or prevent barriers from occurring. Suppose the onset of a barrier cannot be prevented. In that case, an organization should track the barriers their employees commonly experience and use that information to inform the revision of future training. In addition, an organization may consider providing a resource of potential strategies that can be used to address or mitigate barriers that arise. For example, an organization could create or modify an existing troubleshooting guide (see Lee et al., 2015, for an example) that includes a table or decision-making tree of steps the supervisor can engage in to try to address or mitigate issues that arise.

Furthermore, an organization may consider investing in and providing their employees with appropriate and adequate resources, especially technology-related, to help offset barriers they may experience. Investments in technology may be especially important, given our findings that internet connectivity and obstruction of view are common barriers our participants reported. Finally, we recommend that the organization continuously tracks barriers that their employees experience to inform quality improvement and revisions to training and resources already available.

Individual Level

Our results suggest that internet connectivity is a persistent barrier to supervision via telehealth. Therefore, before beginning the supervision process, we recommend that the supervisor and supervisee acquire internet and ensure that internet connections are strong. Suppose the supervisor or the supervisee cannot acquire internet or strong internet connections in their current settings. In that case, the supervisor and supervisee may consider lobbying their organization to upgrade their internet service or reduce the number of devices connected to the internet (Lee et al., 2015). If acquiring internet is still unsuccessful, they may consider changing their location to one where the internet is available, and the connection is strong, using an asynchronous modality, or using a different modality (e.g., phone instead of a computer; Neely et al., 2022).

Communicating the expectations of supervision from the beginning of the supervision process may increase the effectiveness of supervision (Sellers et al., 2019) and lead to continued growth and development for both the supervisee and supervisor (Valentino, 2021). Therefore, the supervisor should clarify and set clear expectations (e.g., how and when feedback will occur, how to receive feedback) for both the supervisee and supervisor. When clarifying and setting expectations, the supervisor should also include information about how supervisees are to engage in professional development activities (e.g., conferences; see Becerra et al., 2020), complete assignments by predetermined deadlines (e.g., prior to weekly supervision meetings), and how to apply what they learned during supervision to their practice (e.g., how to implement feedback; Valentino, 2021). In addition, supervisors should consider having a formal conversation with the supervisee about appropriate places to hold supervision meetings (e.g., avoid public places or wear headphones if public places cannot be avoided; Britton & Cicoria, 2019).

If the supervisor is unfamiliar with technology or video conferencing software, they should first obtain training on how to use these resources (Ninci et al., 2021). In addition to reading or reviewing any manuals or instructions accompanying that technology or software, the supervisor should ask for guidance from someone within their organization who may be familiar with using these resources. If the supervisee is unfamiliar with technology and video conferencing software, the supervisor may consider providing training to the supervisee (Ninci et al., 2021).

In addition, suppose the supervisor provides training to the supervisee on using technology and video conferencing software. In that case, the supervisor may consider including a discussion about how to ensure ethical considerations and practices (e.g., how to manage data appropriately) are adhered to when providing supervision via telehealth (Britton & Cicoria, 2019; Cavalari et al., 2015; Quigley et al., 2019). This is especially important given the results of the present study, in which 12 participants indicated they used a variety of technologies when providing supervision. The supervisor and supervisee should ensure privacy and confidentiality are protected by only using technology that is HIPAA or FERPA compliant (Pollard et al., 2017). The supervisor may also consider providing the supervisee with any additional resources they could access when encountering technology issues throughout the supervision process (e.g., a troubleshooting guide for the video conferencing software, task analysis for how to set up and tear down a webcam; see Zoder-Martell et al., 2020, for an example).

Methodological Contributions

Though the primary purpose of this survey was to inform practice recommendations, this study also extends the behavior-analytic research literature through its survey methodology. To begin, we reported the participant response rate, which is necessary for understanding the extent to which the results represent the study’s sample. When designing this study, we noticed that the frequency of survey studies in behavior-analytic journals reporting response rates was low. Therefore, the first author conducted a cursory review of articles published in Behavior Analysis in Practice and the Journal of Applied Behavior Analysis between 2017 and 2020 before distributing the survey reported in this study. From those two journals, seven survey studies were identified. Of the seven studies, five (i.e., Brown et al., 2020; Colombo et al., 2020; Hajiaghamohseni et al., 2020; Richling et al., 2019; Saini et al., 2017) recruited their participants through the BACB mass email service and did not report a response rate. One study (i.e., Sellers et al., 2019) recruited participants through the BACB mass email service and various social media sites (e.g., the Facebook site for the Association of Behavior Analysis International). Finally, one study (i.e., Brand et al., 2020) recruited participants through agencies providing services to individuals with disabilities. Neither Sellers et al. (2019) nor Brand et al. (2020) reported a response rate. Therefore, a major limitation of the beforementioned studies is that they are unclear to what extent their results represent the relative samples.

Because of our low response rate, our results did not have the statistical power to allow us to explore supervisee or supervisor variables that may predict specific barriers. However, our study provides a framework for future researchers to explore such predictors. Because our survey and accompanying materials are freely available in an open science repository, our study presents a readily available template from which future researchers can evaluate the extent to which correlations exist between supervisor or supervisee variables and the barriers they experience. Such information, we believe, would not only result in a more rigorous analysis of survey results but also enhance the field’s understanding of factors that may affect supervision delivered via telehealth and may also lead to more direct manipulation (e.g., in a single case study) of independent and dependent variables that improve supervisory practices. Related to this, we hope that by making the entirety of our method and results publicly available, we can inspire behavioral researchers to increase the transparency of their research activities further.

Limitations

Several limitations of the present study should be noted. First, the reliability and validity of the survey were not evaluated. However, expert and content reviews were conducted on the survey. Future survey research should consider evaluating the reliability and validity of their survey prior to distributing it to minimize measurement error (Alwin, 2010). A second limitation was the low response rate of 0.8%. A low response rate affects an adequate sample size and a representative sample (Krezmien et al., 2017). As a result, the low response rate and small sample size limit the generalization of the results to all BCBAs and BCBA-Ds providing supervision via telehealth.

In addition, the low response rate and small sample size limit the ability to conduct additional statistical analyses (e.g., logistic regression of barriers and strategies) to further explore and analyze the research questions (Rogelberg & Stanton, 2007). A potential cause for the low response rate in the present study could be survey fatigue experienced during the COVID-19 pandemic. Research has shown that during a public health emergency (e.g., the COVID-19 pandemic), the overexposure to online and telephone surveys results in individuals becoming fatigued and not participating in surveys (Field, 2020; Patel et al., 2020). Given that the survey of the present study was distributed during the COVID-19 pandemic, it could be possible that potential participants had been receiving numerous survey requests, resulting in survey fatigue.

Because the survey was open and available for participants to complete for only 2 weeks, it is possible that potential participants were missed, which may have also contributed to the low response rate. Potential participants could access the survey only once using the link provided in the email to prevent individuals from submitting multiple responses. Therefore, it is possible that potential participants were missed if they had exited out of the link and could not reaccess the survey. Participants consisted of only individuals with a BCBA credential or a BCBA-D who provided supervision via telehealth to supervisees. Therefore, we only received responses from the supervisor. The extent to which supervisee and supervisor responses positively correlate is unknown.

Participants consisted of BCBAs who provided supervision via telehealth to another BCBA or an individual pursuing a BCBA credential (i.e., trainee). This may be problematic as BCBAs who provided supervision via telehealth to another BCBA may have experienced different barriers than BCBAs who provided supervision via telehealth to an individual pursuing a BCBA credential. Future research should ask participants to indicate for whom they supervise and analyze the data separately. We also did not collect information about the location where supervision was provided, each participant’s position within the organization, or caseload size. Finally, when asking participants to indicate what strategies (if any) they used to address or mitigate each barrier, participants selected the strategies used. As a result, it is unclear if the strategies used effectively addressed or mitigated the barriers. Future research could evaluate the effectiveness of strategies used to address or mitigate supervisee and supervisor barriers to help inform the development of training and resources for individuals providing supervision via telehealth.

A Call for Direct Empirical Research

In this discussion, we highlighted several areas for future research in supervising behavioral interventions. We describe this future research in various forms, ranging from survey research, which provides meaningful information about its sample and informs direct and more controlled experimental arrangements, to direct and controlled experimental arrangements. Because the field of ABA now has an established body of literature describing practical recommendations for supervision, it is time to shift towards direct empirical evaluations of recommendations made in discussion-based (e.g., Brodhead & Higbee, 2012) or survey-based articles (e.g., Sellers et al., 2019; Kranak et al., 2023). After all, a controlled preparation that confidently demonstrates a functional relation (or lack thereof) is the best way to determine if a specific intervention works in a specific context. Given the field’s long history of conducting single-case research, behavior analysts are uniquely positioned to conduct such evaluations, as those single-case arrangements translate directly to and subsequently inform practice-based decisions. The results further supported the need for empirical studies within supervision research and the call for action by Kranak et al. (2023). The field of ABA must begin to evaluate available tools and resources and use data from those evaluations to make improvements or changes to those tools and resources, if necessary.