Many individuals with and without disabilities engage in problem behavior (e.g., aggression, disruptions, self-injurious behaviors) that can impede their ability to participate in everyday life at home, school, work, and in the community. One way to decrease problem behavior is to identify variables maintaining their occurrence using a functional analysis. A functional analysis is an experimental manipulation of the antecedents and consequences surrounding a certain behavior and is the only method that allows researchers and practitioners to confirm hypotheses about the function of their client’s problem behavior. Once the function (i.e., cause) of behavior is determined, the behavior analysts can develop interventions to decrease the problem behavior and increase more socially appropriate behaviors that serve the same purpose as the problem behavior (Cooper et al., 2007).

Iwata et al. (1982/1994) developed the standard or traditional functional analysis (FA), which included three experimental test conditions: social disapproval, academic demands, and alone. A control condition, called unstructured play, was also used as a comparison wherein all reinforcers were available regardless of behavior, reducing motivation to engage in problem behavior. Thereafter, social disapproval, academic demands, alone, and unstructured play conditions became attention, escape, alone, and toy play conditions, respectively (Cooper et al., 2007). Additionally, researchers have modified the procedures described by Iwata et al. in various other ways.

Modifications to the Iwata FA have included the brief functional analysis (Northup et al., 1991), the trial-based functional analysis (Sigafoos & Saggers, 1995), and the precursor functional analysis (Heath & Smith, 2019), among others. Furthermore, researchers have modified the original conditions (e.g., divided attention; Mace et al., 1986) and have tested for social functions other than attention and escape (Hanley et al., 2003). Examples of other common conditions include the tangible (Mace & West, 1986), and social avoidance (Slocum et al., 2021) conditions. Idiosyncratic conditions, such as access to preferred conversation topics (Roscoe et al., 2010), or escape from various transition types (McCord et al., 2001) have also been demonstrated to produce valid outcomes.

Another modification used by researchers includes test conditions for synthesized functions. Synthesized “refer[s] to arrangements that involve multiple EOs [i.e., establishing operations], multiple potential reinforcers, multiple response topographies, or some combination” (Slaton & Hanley, 2018, p. 945). When all three are combined, it is a synthesized contingency (Slaton & Hanley, 2018). Slaton and Hanley (2018) reported that researchers have compared synthesized and isolated conditions (i.e., those with only one EO/reinforcer) in roughly 30 applications. Slaton and Hanley (2018) determined that synthesis was necessary to show differentiated results in an FA or effective treatment in 80% of those applications. Hanley et al. (2014) developed one application of synthesized contingencies when they used an interview to create a single test condition with synthesized EOs. A matched control condition (i.e., in which the researchers provided continuous access to the reinforcers used during the test condition) was used to test synthesized contingencies for three participants. Henceforth, this procedure has been known as an interview-informed synthesized contingency analysis (IISCA).

Hanley et al. (2014) assessed the problem behavior of three individuals aged 3, 8, and 11 years old, diagnosed with autism or PDD-NOS, who were referred to a university-based clinic for services. All three participants exhibited loud vocalizations, disruptions, and aggression. The researchers interviewed each participant’s parent using the open-ended Functional Assessment Interview (FAI), which provided demographic information and informed the researchers about possible antecedents and consequences of the participant’s problem behavior (Hanley, 2012). The researchers then conducted an unstructured observation in which they presented and removed various stimuli (e.g., toys, attention, demands) and noted the results (Hanley et al., 2014).

Test conditions for the IISCA were developed from the interviews and direct observations and included a single, synthesized condition to test all potential functions for each participant. At the beginning of the test condition sessions, the researchers provided access to all reinforcers for 30 s and then removed them. The reinforcers were returned for 30 s only when the participant engaged in problem behavior. During control conditions, the reinforcers were freely provided for the entire session, regardless if problem behavior occurred. The researchers alternated all participants’ test (T) and control (C) sessions following a fixed pattern of control-test-control-test-test (CTCTT) for the first five sessions. Results indicated that all participants showed differentiated responding in their synthesized conditions when compared to their corresponding matched controls. That is, relative to control conditions, each participant engaged in more problem behaviors during their synthesized test conditions, resulting in differentiated FAs.

Coffey et al., (2020a, 2020b) completed a literature review on IISCA publications between 2014 and October 2018. Their search criteria included any articles that mentioned the use of the IISCA or the procedures implemented by Hanley et al. (2014). A total of 17 articles across five journals were identified, with 89 participants with 102 IISCA applications. Additionally, 14 studies reported conducting treatments with 55 treatment evaluations (Coffey et al., 2020a, 2020b). They reported participant demographics including age, vocal abilities, diagnoses, and problem behaviors. Additionally, the researchers reported the setting, number of sessions, duration of sessions, and whether analyses were differentiated for each IISCA conducted. For studies that reported conducting treatments, the researchers also looked at the type of intervention and degree of behavior reduction by the end of the treatment. Coffey et al., (2020a, 2020b) did not, however, calculate effect sizes for the IISCAs or treatments identified in their review. Nor did they evaluate the methodological quality of included reviews by assessing if studies met the design standards for single-case research developed by What Works Clearinghouse (Coffey et al., 2020a, 2020b; What Works Clearinghouse, 2020).

Relative to the broader functional analysis literature, the IISCA literature has appeared more recently, includes a limited number of studies, and there has not been a systematic review of the IISCA literature that includes effect size calculations to summarize the effect of IISCA derived treatments. Therefore, the purpose of the current study is to add to the IISCA literature by conducting an updated review of the literature so that more studies may be included. Additionally, this study will include a meta-analysis of the studies that include IISCA derived treatments so that an overall effect of IISCA derived treatments can be estimated. Finally, this study includes a review of the methodological rigor of research designs used to test IISCA derived treatments, which has not been included in previous reviews. This study will make an important contribution to the IISCA literature and identify gaps in the literature and future research directions.

The following research questions were also used: (1) Does the IISCA produce differentiated results? (2) Do function-based interventions developed from the results of IISCAs produce meaningful reductions in destructive problem behaviors? (3) To what extent do function-based interventions developed from the results of IISCAs meet research design standards as defined by What Works Clearinghouse? (4) What modifications have been to the procedures found in the original IISCA study (Hanley et al., 2014)? (5) What are the demographics of participants included in IISCA research?

Method

Search Process

Article Identification

During the current review, the researchers followed 70% of the Preferred Reporting Items for Systematic reviews and Meta-Analyses guidelines (Page et al., 2021). Figure 1 illustrates the search process. To identify articles, the researchers conducted a search using the PSYCHinfo and ERIC databases (last searched 5/15/2022). The researchers used the following search terms in the first line: “interview-informed synthesized contingency analysis” OR “IISCA” OR “Practical Functional Analysis” OR “PFA”; “functional analysis” OR “FA” OR “functional behavior assessment” OR “FBA” OR “functional assessment” on the second line; and “synthesized” OR “multiple control” OR “multiple reinforcers” OR “combined reinforcers” OR “combined + problem behavior” OR “multiple + problem behavior” OR “synthesized contingency” on the third line. The first and second lines were connected by the “OR” Boolean operant, while the second and third lines were connected by the “AND” Boolean operant. The initial search generated 346 articles. As Hanley et al. (2014) published the original IISCA research study in 2014, the researchers applied a year limitation to only include studies published in 2014 or later. Following the year limitation, 140 articles were removed, leaving 206 studies.

Fig. 1
figure 1

Literature search process. This figure illustrates the inclusion and exclusion of articles during the literature search process

Full size image

Abstract and Title Review

The researchers screened the 206 remaining articles via a title and abstract review. During the title and abstract review, the researchers excluded articles if it was evident that they met at least one of the following exclusion criteria: (a) the article was written in any other language than English, (b) behavior was not the dependent variable, (c) participants were not human, (d) data in the article were not original research (i.e., all data had been included in a previous article or the article was a meta-analysis, literature review, or systematic review), (e) article was published before 2014 (i.e., before the Hanley et al., 2014 article was published), or (f) no functional analysis was conducted. Following the title and abstract review, 159 articles were removed, leaving 43 articles that were retained.

Full-Text Review

After the title and abstract review, the researchers conducted a full-text review of the remaining 43 articles. The researchers used the following inclusion criteria during the review (i.e., articles had to meet all inclusion criteria): (a) article was written in English, (b) behavior was the dependent variable, (c) research was original (i.e., not previously published or articles is a systematic review/meta-analysis/literature review), (d) article was published in 2014 or after, (f) a synthesized contingency analysis was conducted, (g) the Hanley (2012) interview was conducted to inform the functional analysis, (h) a single-case design was utilized in the functional analysis or treatment analysis. Following the full-text review, 17 articles were removed, leaving 26 articles.

The researchers excluded 12 studies during the full-text review as they contained neither the Hanley (2012) interview nor a synthesized contingency analysis. Specifically, three articles did not include the Hanley (2012) interview, while two studies did not conduct a synthesized contingency analysis. Additionally, the researchers excluded eight studies as they were not original research using a single-case design, and one article was excluded because the participants were not human. The researchers also removed four duplicate articles during the full-text review stage. Additionally, if a dissertation/thesis was published on the same data set as a peer-reviewed journal article, the journal article was retained, and the dissertation or thesis was removed.

Ancestral and Descendant Citation Search

For each article that passed the full-text review, the researchers conducted ancestral and descendant citation searches to identify any potentially missed articles not identified by the initial literature search. For the ancestral citation search, the researchers reviewed the titles of all articles listed in the references section. Additionally, for the descendant citation search, the researchers used Google Scholar to review articles that cited each article that passed the full-text review. The researchers followed the same criteria listed for the title and abstract review and full-text reviews for any potential articles. Following the ancestral and descendant searches, an additional ten articles were identified, resulting in 36 articles following this stage.

The researchers also created alerts on Google Scholar for “interview-informed synthesized contingency analysis” and “practical functional assessment” to identify additional articles. The researchers identified an additional three articles using these Google Scholar alerts. This resulted in a total of 39 articles that were included in the current review.

Variable Coding

For each participant, the researchers coded 28 items including each participant’s pseudonym, gender, ethnicity/race, age, verbal ability, diagnoses, and problem behaviors (i.e., topography). They also coded the interviewer and interviewee for each participant’s open-ended functional assessment interview (Hanley, 2012) and whether a direct observation was conducted. Additionally, for each synthesized contingency analysis, the researchers coded the setting, dependent variable measurement system, hypothesized functions, precursor behaviors, implementer, implementer training, session length, and the number of sessions as well as if any modifications were made, whether the SCA followed the CTCTT sequence, if the SCA was reported to be differentiated, whether interobserver agreement (IOA) and procedural integrity were reported, and if treatment was conducted using the results of the IISCA. If IOA was reported, the researchers coded the percentage of sessions with IOA and the average IOA value. If procedural integrity was reported, the researchers coded the method for collecting procedural integrity data, the percentage of sessions with procedural integrity, and the average procedural integrity value.

For participants who received treatment, the researchers coded an additional 16 items for a total of 44 items. These additional items included the type of intervention conducted, single case design used, implementer, implementer training, and whether interobserver agreement and treatment integrity was reported. Again, if IOA was reported, the researchers coded the percentage of sessions with IOA and the average IOA value. If treatment integrity was reported, the researchers coded the method for collecting treatment integrity data, the percentage of sessions with treatment integrity, and the average treatment integrity value. Furthermore, for the methodological quality analysis, the researchers coded whether treatments met the systematic manipulation, interobserver agreement, attempts of intervention, and phase length requirements as outlined by the WWC (What Works Clearinghouse, 2020) research design standards.

Outcomes Analysis

Data Extraction

The researchers completed outcomes analyses for all published graphs for levels of problem behavior during participants’ SCA and treatment analyses. Due to the nature of the data extraction and outcome analysis method, only data displayed using line graphs were included in the outcome analyses. Therefore, the researchers extracted X and Y coordinates for each data point depicting problem behavior from every line graph published in the articles using the DigitizeIt Version 2.5 (Bormann, 2012) software. DigitizeIt was deemed a reliable and valid method for extracting raw data from single-subject experimental research (Rakap et al., 2016).

Kendall’s Tau

Following extraction, the researchers input the Y-coordinates for different conditions (e.g., test vs. control; baseline vs. treatment) into a Tau-U calculator (Vannest et al., 2016) to calculate a Kendall’s Tau coefficient for SCA and treatment graphs. The researchers calculated a Tau-U coefficient for each set of adjacent control/baseline (A) and test/treatment (B) conditions (Parker et al., 2011) within all applicable SCA and treatment graphs. Therefore, the researchers calculated a Tau coefficient between the control (A phase) and test (B phase) conditions for each synthesized contingency analysis. If a participant had participated in multiple SCAs, a Tau-U coefficient was conducted separately for each SCA. The researchers also calculated a Tau-U coefficient for each treatment graph for every adjacent baseline (A) and treatment (B) condition.

Hedge’s g

The researchers also used extracted data to calculate omnibus Hedge’s g coefficients for the IISCA and the treatments designed from the results of IISCAs. Several assumptions must be met to include studies and condition comparisons in the Hedge’s g calculation. For example, one Hedge’s g assumption is that the article provided data for at least three participants. Therefore, any studies that only included one or two participants were excluded from the calculations. Additionally, the standard deviations for conditions had to be larger than zero. Therefore, any condition comparisons in which at least one of the standard deviations (i.e., for the baseline or treatment conditions) was zero were excluded from the calculations as well. The researchers calculated separate Hedge’s g coefficients for SCAs and treatment graphs.

IOA

Two researchers conducted interobserver agreement (IOA) during the article search process, variable coding, and data extraction phases for at least 20% of articles. The researchers calculated IOA via a trial-by-trial method during the initial literature database, title and abstract review, and full-text review (Cooper et al., 2007). An agreement was counted if the results of the initial literature search of both researchers provided the same article(s) or both researchers retained the article in the title and abstract review or full-text review. Then, the number of agreements was divided by the total number of agreements plus disagreements. The outcome was then multiplied by 100% (Cooper et al., 2007). A disagreement was counted if one researcher’s initial literature database results provided an article that was not included in the other researcher’s results or one researcher retained an article while the other researcher did not (Cooper et al., 2007). If there was a disagreement on an article, both researchers met to decide on a consensus on whether the article met inclusion criteria.

The researchers calculated IOA for variable coding and data extraction using a mean count per interval IOA method. IOA was calculated for each variable by comparing the codes of each researcher. If the researchers both had the exact same codes, that variable had an IOA of 100%. If both researchers did not put the same code (e.g., one researcher put “2” while the other put “3”), IOA for that variable was 0%. If the researchers had some, but not all, of the same codes IOA for that variable was calculated. For example, if one researcher put “2” while the other put “2;3,” they agreed on one of two codes. Therefore, IOA for that variable would be 50%. The values for each code were averaged for each participant. Similarly, IOA was calculated for each data point in data extraction by dividing the smaller value (i.e., extracted by one researcher) by the larger value (i.e., extracted by the other researcher). These values were then averaged and multiplied by 100% to provide an overall IOA value for data extraction (Cooper et al., 2007).

Results

The researchers identified 39 studies that included IISCA analyses or treatments. Twenty-nine studies published both SCA and treatment analyses, while nine studies published SCA analyses alone. Additionally, one article, Ward et al. (2021), published only treatment analyses; however, the SCA analyses for the participants in Ward et al. (2021) were published in a previous article, Warner et al. (2020). See Table 1 for study-level information, including the number of participants, average Tau scores for SCA and treatment analyses, and whether treatment analyses met WWC design standards (What Works Clearinghouse, 2020). The 39 studies were published between 2014 and 2022 across 10 different peer-reviewed journals including the Journal of Applied Behavior Analysis (n = 17), Behavioral Intervention (n = 6), Behavior Analysis in Practice (n = 4), Education & Treatment of Children (n = 2), Advances in Neurodevelopmental Disorders (n = 1), Behavior Modification (n = 1), the Canadian Journal of School Psychology (n = 1), Developmental Neurorehabilitation (n = 1), the European Journal of Behavior Analysis (n = 1), and the Journal of Autism and Developmental Disorder (n = 1). Figure 2 demonstrates the cumulative frequency of articles published per year and indicates a steady increase in the number of articles published on the IISCA since 2017. Within those 39 studies, 235 participants completed 293 SCA and 111 treatment evaluations.

Table 1 Study level information including overall treatment quality and outcome analyses
Full size table
Fig. 2
figure 2

Cumulative frequency of IISCA articles from 2014 to 2022

Full size image

Participant, Functional Analysis, and Treatment Characteristics

Participant Characteristics

Participant characteristic items are summarized in Table 2. Overall, participants included in IISCA research tended to be male (80%) with a mean age of 7.2 (range, 1–35). Ethnicity/race was not reported for most participants (86%); however, when it was reported, participants tended to be identified as White or Caucasian (9%) rather than people of color (i.e., Hispanic/Latino, Black/African American, and Asian; 6%). Verbal abilities amongst participants were variable, but the highest percentage of participants were fully fluent (31%).

Table 2 Participant characteristics
Full size table

Additionally, 87% of the participants had a diagnosis, while 13% did not. Additionally, approximately 76% of participants had a diagnosis of autism spectrum disorder. Many participants also had a diagnosis of intellectual disability (20%) or ADHD (16%). The top three reported topographies of problem behavior included aggression (82%), disruptive behaviors (44%), and self-injurious behaviors (41%).

Functional Analysis Context

Functional analysis context items are summarized in Table 3. As four participants were included in two entirely independent functional analyses, including separate interviews and direct observations, there were 232 functional behavior assessments (i.e., functional analysis context combined with synthesized contingency analyses). Approximately half of the Open-Ended Functional Assessment Interviews (Hanley, 2012) were conducted by a behavior analyst or therapist (51%) while the most common interviewee was the participant’s caregiver (72%). The researchers conducted a direct (i.e., descriptive) observation before the synthesized contingency functional analysis with 80% of participants.

Table 3 Functional analysis context
Full size table

Synthesized Contingency Analysis

Because many participants were included in multiple analyses, there were 293 SCAs completed across the 235 participants. SCA results are summarized in Table 4. Seventy-three percent of participants’ analyses were conducted in an outpatient or university-based clinic. Frequency or rate was used to measure the dependent variable, problem behavior, in the functional analysis for 95% of analyses. In each analysis, all hypothesized functions were combined into one test condition for each participant’s SCA. Eighty-one percent of SCAs included an escape function, 58% included an attention function, 95% included a tangible function, and 13% included mand or request compliance as a function. Precursor behaviors were also included in 16% of analyses while 10% of studies included precursor behaviors in a separate or second functional analysis.

Table 4 Synthesized contingency analysis results
Full size table

A large majority (93%) of SCAs were implemented by a behavior analyst or behavior therapist (including those identified as a BCBA or BCBA-D). Training for 84% of implementers was not reported, although 15% of implementers were reported to have received prior training on conducting functional analyses. The mean, as well as the mode, session length (45%) was 5 min. Similarly, the average number of sessions included in each SCA was 6 (range, 1–21), while the mode was 5 (50%). Forty-two percent of SCAs included at least one modification to the original Hanley et al. (2014) procedures (e.g., including less or more than 5 sessions, switching implementers, changing the contingencies or functions included). The control-test-control-test-test sequence used by Hanley et al. (2014) was also used by 67% of analyses in this review. Overall, 96% of SCAs were reported to be differentiated by authors of the studies.

IOA was reported for nearly all (99.66%) analyses. Additionally, 93% of analyses reported collecting IOA in at least 20% of SCA sessions. The average IOA value was 96% (range, 85–100%). However, procedural integrity was only reported for 23% of analyses. For those analyses that reported procedural integrity, all analyses used direct observation to collect procedural integrity data. All analyses that reported procedural integrity collected it during at least 20% of sessions. The average procedural integrity value was 98%.

Treatment

A treatment was conducted for 46% of participants in the current review. Treatment results are summarized in Table 5. Ninety-five percent of treatments used functional communication training with a total of 64% of treatments using functional communication training plus delay and denial tolerance. The most common single case design used to test the efficacy of these treatments was a changing criterions design (51%), while withdrawal/reversal designs and multiple baseline/probe designs were also common (22% and 24%, respectively). Ninety percent of treatments were implemented by a behavior analyst or therapist. Similar to the SCAs, training was not reported for 87% of treatment implementers.

Table 5 Treatment results
Full size table

IOA was reported for 100% of the treatment analyses with 100% of the analyses having IOA collected in at least 20% of sessions. The average IOA value for the treatment analyses was 97% (range, 85–100%). Treatment integrity was only reported for 36% of treatment analyses. Direct observation was used to collect the treatment integrity data for all the analyses that reported treatment integrity. Additionally, treatment integrity was collected in at least 20% of all analyses that reported treatment integrity. The average treatment integrity value was 98% (range, 95–100%).

Methodological Quality Analysis

Methodological Quality results are summarized in Table 6. Overall, 8% of treatments met all four What Works Clearinghouse (WWC) research design standards without reservations, 41% met WWC standards with reservations, and 37% did not meet WWC standards. The researchers could not determine if 14% treatment analyses met WWC design standards due to the researchers being able to assess the number of data points per phase. However, 100% of treatments met the systematic manipulation design standard and 77% of the treatments met the IOA design standard. 95% of the treatments met the attempts of intervention design standard, 3% did not, and 3% could not be determined. 10% of treatments met the phase length standard without reservations, 44% met with reservations, and 12% did not meet standards. Whether analyses met the phase length design standard could not be determined for 34% of treatment analyses.

Table 6 Methodological quality results
Full size table

Outcome Analysis

The researchers extracted X and Y coordinates from every SCA and treatment line graph published in the included studies. Unfortunately, participants’ SCA analyses in Curtis et al. (2020) and treatment analyses in Gover (2020) were not displayed as line graphs, and therefore, data could not be extracted for these participants. Likewise, SCA graphs for some participants (n = 22) in Jessel et al. (2021) were not published, and Sidwell et al. (2021) did not publish graphs for SCA analyses or treatment analyses for any participant. Additionally, Fiani and Jessel (2022), Jessel et al., (2018a, 2018b, 2018c), and Rajaraman et al. (2022) published treatment data for only some of their participants. Therefore, the researchers only extracted data for 259 of the 293 SCA analyses and 73 of the 111 treatment analyses.

Tau-U

A summary of the outcome analyses for the SCA and treatment analyses are displayed in Figs. 3 and 4, respectively. Two hundred fifty-nine Tau coefficients were calculated for SCAs and ranged from − 0.33 to 1.0. On the other hand, as many treatments were conducted with more than one baseline condition, 263 pairs of baseline and treatment conditions were able to be extracted from the treatment analyses. Therefore, 264 Tau coefficients were calculated for treatment analyses and ranged from − 0.71 to 1.0. Average Tau coefficients per study were calculated and are displayed in Table 1.

Fig. 3
figure 3

Summary of Tau-U coefficients for synthesized contingency analyses

Full size image
Fig. 4
figure 4

Summary of Tau-U coefficients for treatment analyses

Full size image

Hedge’s g Effect Size

An omnibus Hedge’s g coefficient was calculated for the SCAs and treatment analyses. The omnibus effect size using Hedge’s g for the SCAs was 2.427, p < 0.0001, which is considered a large effect size. However, 12 of the 28 IISCA studies (i.e., Beaulieu et al., 2018; Boyle et al., 2019; Coffey et al., 2020a, 2020b; Dowdy & Tincani, 2020; Ferguson et al., 2020; Ghaemmaghami et al., 2015; Herman et al., 2018; Jessel et al., 2018a, 2018b, 2018c; Rose & Beaulieu, 2019; Santiago et al., 2016; Strand & Eldevik, 2016; Taylor et al., 2018) were excluded as they did not provide data for at least 3 participants. Additionally, 184 IISCA comparisons were not included in the Hedge’s g calculation as their standard deviation for at least one of the conditions (i.e., control or test) was zero (i.e., all data points in the condition were the same value).

The omnibus effect size for the treatment analyses was 2.007, p < 0.0001, which is also considered a large effect size. Similarly, 13 of 29 articles (i.e., Beaulieu et al., 2018; Boyle et al., 2019; Coffey et al., 2020a, 2020b; Dowdy & Tincani, 2020; Ferguson et al., 2020; Fiani & Jessel, 2022; Ghaemmaghami et al., 2015; Herman et al., 2018; Rajaraman et al., 2022; Rose & Beaulieu, 2019; Santiago et al., 2016; Strand & Eldevik, 2016; Taylor et al., 2018) that conducted a treatment analysis were not included in the omnibus Hedge’s g calculation for treatments analyses as they did not publish data for at least 3 participants. Furthermore, 32 treatment analyses were not included as one the condition’s standard deviations was zero.

IOA

Search Process

Two researchers conducted the initial literature database search to confirm the primary researcher’s reliability of the initial literature database search. IOA was collected on whether the second researcher’s search included the first 170 articles (i.e., out of 346; 49%) displayed in the primary researcher’s search. All 170 articles were included in both researchers’ database searches, resulting in 100% IOA for this step.

Two researchers independently conducted the title and abstract review for all 346 articles (i.e., 100% of articles) generated from the initial database literature search. Both researchers agreed on whether 199 articles passed (i.e., or did not pass) the screening criteria. However, for 7 articles, one researcher decided that the article passed the screening criteria, while the other did not. Therefore, IOA for this step was 96.60%.

Additionally, two researchers independently conducted the full-text review of the 63 articles (i.e., 21%; selected at random) identified via the database search (n = 43), citation searches (n = 17), and other sources (n = 3). Researchers agreed on whether an article met inclusion criteria for 12 articles, while they disagreed on 1 article. For the one disagreement, the two researchers met and decided to follow the primary researcher’s decision to exclude the article. IOA for the full-text review was 92%.

Variable Coding

Two researchers independently completed the variable coding for 8 of 39 articles (i.e., 21%). The researchers calculated IOA for variable coding using a mean count-per-interval method for each participant on each item coded. IOA was calculated for each variable and then averaged. IOA was 86.60% (range: 72.22–94.44%) for the variable coding.

Outcomes Assessment-Data Extraction

Two researchers independently completed the raw data extraction for 10 of 37 articles (i.e., 27%) that had data extraction for SCA graphs, with seven articles out of 27 (i.e., 26%) including treatment analyses. IOA averaged 97.67% (range: 87.88–99.75%) for SCA graphs and 96.74% (range 89.55–99.65%) for treatment graphs.

Discussion

The IISCA was developed by Hanley et al. (2014) and provides an efficient, standardized method to assess synthesized functions of behavior. Function-based interventions developed from the results of IISCA appear to lead to decreased levels of problem behavior. To date, thirty-nine studies have been published in which an IISCA was conducted. This included 29 studies that also implemented a function-based intervention developed from the results of an IISCA. The purpose of the current study was to systematically review the literature on the IISCA and its subsequent treatments and test the effectiveness of the IISCA to provide differentiated results and effective treatments across various participants, settings, and procedures.

Research Questions

Question 1: Does the IISCA Produce Differentiated Results?

Previous reviews on functional analyses have reported that approximately 94% of functional analyses published in peer-reviewed journals, including synthesized functional analyses, have produced differentiated results (Beavers et al., 2013; Slaton & Hanley, 2018). Using visual analysis methods (i.e., visually analyzing data based on the trend, level, and variability of data; Gast & Ledford, 2014), 96% (n = 280) of published IISCAs in the current study were reported to be differentiated. This indicates that the IISCA is equally, if not more, likely to produce differentiated results compared to functional analyses in general.

Similarly, results of the Tau-U calculations show that 93% of IISCAs included in the outcomes analysis produced large to very large effect sizes (i.e., Tau-U coefficients of 0.6 to 1.0). Conversely, 96% produced moderate to very large effect sizes (i.e., Tau-U coefficients of 0.2 to 1.0). Overall, the average Tau-U coefficient for the IISCA in the current study was 0.92, which represents a large to very large effect size. Results of the omnibus effect size calculation provided a Hedge’s g of 2.428 which indicate a large effect size for the IISCA as a whole. However, keep in mind that these statistics only refer to those IISCAs that were published in peer-reviewed journal articles or as dissertations or theses. The actual effect sizes of all IISCAs that have been conducted are unknown.

Question 2: Do Function-Based Interventions Developed from the Results of IISCAs Produce Meaningful Reductions in Destructive Problem Behaviors?

Reductions in the levels of problem behavior were seen in all 111 published treatment analyses, regardless of treatment modality, and identified functions of behaviors. On average, behavior was reduced by 97% (range 60–100%) when the levels of problem behavior in the last five treatment sessions were compared to baseline levels. Thirty-five (31%) treatment analyses showed a 100% reduction in problem behavior. Additionally, 92% of Tau-U coefficients were indicated in the large to very large effect size range (i.e., coefficients between 0.6 and 1.0), while 98% of coefficients were in the moderate to very large effect size range (i.e., 0.2 to 1.0). The overall average Tau-U coefficient for treatments developed from the results of IISCAs was 0.89, indicating a large effect size. Similar coefficients were seen with the omnibus Hedge’s g of 2.007 which indicates a large effect size.

These results are similar to an average Tau-U score of 0.86 calculated by Walker et al. (2018) in their review of function-based interventions in schools. Additionally, effect size calculations were completed by Slaton and Hanley (2018) in which the percentage of nonoverlapping data (PND) was 88.60%. Therefore, treatments developed from the results of IISCA are comparable to other function-based interventions such as synthesized treatments and function-based interventions in schools. Again, these values only represent published treatments. Therefore, the effect sizes of treatments developed from the results of IISCAs in clinical practice are unknown.

Questions 3: To What Extent Do Function-Based Interventions Developed from the Results of IISCAs Meet Research Design Standards as Defined by What Works Clearinghouse?

Only 8% of treatment analyses (n = 9) met all four research design standards without reservations. Forty-one percent of the analyses met with reservations, while 37% (n = 41) of analyses did not meet standards. For those analyses that did not meet design standards, over half of those did not meet the IOA design standard of having IOA collected for at least 20% of sessions in each condition. The systematic manipulation design standard was the only standard met by all 111 treatment analyses. Additionally, 95% of analyses also met the attempts of intervention effect design standard, which required at least three attempts to show the intervention effect. However, only 54% of analyses met design standards (i.e., with or without reservations) for phase length by having at least three data points per condition (i.e., withdrawal, reversal, pairwise, changing criterion designs, multiple baseline, or multiple probe designs) or at least four repetitions of each condition (i.e., for alternating treatments or multi-element designs). The primary therapist could not determine whether 38 analyses met the phase length design standards due to the lack of published graphs for those analyses.

Additionally, one limitation of the IISCA itself is that it does not meet What Works Clearinghouse design standards as it is designed. The IISCA follows a multi-element design which requires five data points per phase to meet design standards without reservations and three data points per phase to meet design standards with reservations. The IISCA is designed to be implemented in five sessions, two control sessions, and three test sessions. Therefore, for the IISCA to meet design standards with reservations, it would need to include at least three control sessions and three test sessions, and five for each to meet design standards without reservations. Therefore, without modifications to the IISCA design, IISCA studies cannot meet all design standards.

Perhaps the most troubling finding in regard to the design standards evaluation is that findings from this study shine a light on the frequent failure of researchers testing IISCA to collect procedural integrity data during IISCA analyses and treatment integrity data during treatment analyses. In fact, just over 77% of IISCA analyses and nearly 64% of treatment analyses did not include procedural integrity or treatment integrity data, respectively. Nearly 30 years ago, Gresham et al. (1993) reported that only 16% of studies published in the Journal of Applied Behavior Analysis between 1980 and 1990 measured and reported accuracy of implementation of the independent variable. Follow-up reviews have found minimal improvement in the extent to which behavior analytic researchers provide empirical evidence for the extent to which the independent variable was implemented as planned (Falakfarsa et al., 2021; McIntyre et al., 2007). Related, this study indicates that approximately 94% of IISCA studies did not report information pertaining to implementer training. As a result, researchers cannot be certain that IISCA analyses or treatments were implemented as intended because there are limited data objectively demonstrating implementation and no description of the rigor with which implementers were trained to implement procedures. In sum, these design flaws constitute monumental threats to the internal validity of IISCA studies. It would certainly behoove behavior analytic researchers to take up the call that Gresham et al. made nearly 30 years ago; that is, provide direct evidence of the extent to which the independent variable was implemented as planned.

Question 4: What Modifications Have Been to the Procedures Found in the Original IISCA Study (Hanley et al., 2014)?

Metras and Jessel (2021) discussed many adaptations to the IISCA that have been made, including a latency-based IISCA, a trial-based IISCA, and a single-session IISCA. However, many studies continue to use the procedures designed by Hanley et al. (2014). Essential elements of the Hanley et al. (2014) IISCA included conducting their IISCAs in an outpatient clinic, having the researchers conduct an open-ended FAI with the participants’ parents, and conducting an unstructured direct observation. Additionally, except where modifications were made for low levels of problem behavior, the researchers in Hanley et al. (2014) had behavior therapists implement the IISCA procedures, collected data on levels of problem behavior measured the frequency of such behaviors, conducted five sessions following a control-test-control-test-test sequence, and did not include precursor behaviors.

Surprisingly, an exact replication of the Hanley et al. (2014) procedures was not observed for any participant in the current review. Despite this, the elements described in Hanley et al. (2014) are still used by most studies. For example, 73.04% of IISCA analyses have been conducted in clinical settings (i.e., outpatient and university-based clinics), and 80.33% of participants participated in a direct observation. Additionally, 93% SCAs were implemented by behavior therapists or analysts (i.e., including BCBAs and BCBA-Ds), and 67% followed the control-test-control-test-test sequence.

On the other hand, just under half (i.e., 42%) of the SCAs included a modification. Specifically, 36% of studies included more than five sessions, while 2% included less than five sessions. Additionally, 35% of SCAs included (or attempted to include) precursor behaviors in the SCA. Furthermore, approximately 2% of SCAs switched contingencies (i.e., the functions they were testing) during the analysis while another 2% switched implementers. Therefore, rather than completing direct replications of the Hanley et al. (2014) article, IISCA researchers are more focused on modifying the procedures to make adaptations that will continue to improve the IISCA and its subsequent treatments.

Question 5: What Are the Demographics of Participants Included in IISCA Research?

Based on the review conducted, participants included in IISCA research tended to be male (80% of participants) rather than female (20%). This proportion of male to female participants is fairly consistent with other functional analysis research. For example, Bruni et al. (2017)’s review on the effects of functional behavior analyses on school-based interventions found that 78% of their participants were identified as male while 22% were identified as female. Similarly, Lloyd et al. (2016) found that 81% of participants in their review of functional analyses conducted in public schools were male while 19% were female.

Additionally, for most participants (86%) in the current review, ethnicity/race was not reported. For participants with an ethnicity/race reported, most participants were White or Caucasian (n = 22) compared to participants of color (i.e., Hispanic/Latino, Black/African American, and Asian, n = 14). Similarly, Severini et al. (2018) found that ethnicity/race was not reported for 64% of participants in their review on problem behavior interventions. Of those participants that had ethnicity/race reported, 45% were identified as Caucasian, 23% were identified as African American; 13% were identified as Hispanic, and 19% were identified as Asian/Indian/Middle Eastern.

This lack of participant ethnicity/race reporting is troubling as an important facet of research evaluation is knowing not only what works, but who does the assessment or intervention work for and under what conditions. Future research must include better documentation of participant demographics so that researchers can better judge the external validity of findings.

Most participants also tended to be preschool or school-aged (i.e., between the ages of 3 and 10; 71%), with a smaller percentage being adolescent or teenager-aged (i.e., ages 11–19; 19%), and an even smaller percentage were adults (i.e., age 20 or older; 3%). Similarly, a small percentage of participants were infants or toddlers (i.e., between 0 and 2; 6%). Similarly, Beavers et al. (2013), combined with data from Hanley et al. (2003), found that 76% of participants were identified as children while 33% were identified as adults in functional analysis research more broadly.

Verbal abilities were somewhat evenly distributed across three of the vocal verbal levels coded (i.e., non-verbal, one-word utterances, and short-disfluent sentences; 15%, 20%, and 20%, respectively), while a larger percentage of participants were reported to be fully fluent (i.e., 31%). However, only a small percentage of participants (10%) were reported to use alternative and augmentative communication (AAC) systems, including speech-generating devices, picture exchange systems, and sign language.

Additionally, 87% of the participants had a mental, behavioral, physical, or cognitive disability, while 13% did not. Additionally, approximately 76% of participants had a diagnosis of autism spectrum disorder. Many participants also had a diagnosis of intellectual disability (20%) or ADHD (16%). On the other hand, fewer participants were diagnosed with a behavioral disorder (5%; e.g., oppositional defiant disorder, conduct disorder) or mood/anxiety disorders (5%; e.g., generalized anxiety disorder, depression, bipolar). In contrast, the Beavers et al. (2013) review found that only 27% of participants were diagnosed with autism spectrum disorder, which is a significantly smaller percentage of participants when compared to the current review. Despite this, Beaver et al. (2013) did find a similar percentage of participants that did not have a diagnosed disability (14%). However, 58% of the participants in Bruni et al. (2017)’s review did not have a diagnosed disability.

Limitations

Several limitations were noted in the current study. First, many articles did not provide graphs for all of their participants’ SCA and treatment analyses. Jessel et al. (2021) did not publish SCA graphs for 22 of 26 participants. Sidwell et al. (2021) did not publish SCA or treatment graphs for their 8 participants. Additionally, Fiani and Jessel (2022), Jessel et al., (2018a, 2018b, 2018c), and Rajaraman et al. (2022) did not publish treatment data for 10 of 11, 22 of 25, or 3 of 4, participants, respectively. Similarly, Curtis et al., (2020; n = 3) and Gover (2020; n = 7) did not display participants’ SCAs using line graphs. Because these articles failed to publish data, 18 and 43 SCA and treatment participants, respectively, could not have data extracted to be included in their respective outcome analyses (i.e., Tau-U and Hedge’s g coefficient calculations). Additionally, the researchers could not determine if these treatment analyses met What Works Clearinghouse designs as they could not assess how many data points were in each phase. Therefore, outcome analyses were only performed using data from 88% of IISCA analyses (i.e., 259 of 293) and 66% of treatment analyses (i.e., 73 of 111).

Furthermore, due to the assumptions needed to calculate an omnibus Hedge’s g for the IISCA and treatments developed from the results of IISCAs, many additional articles and participants (i.e., in addition to participants that did not have their data displayed via a line graph, n = 18) were excluded from the Hedge’s g analyses. For example, 12 of the 38 IISCA studies (32%) and 13 of the 29 treatment analysis studies (45%) were excluded as they did not provide data for at least 3 participants. Additional IISCA comparisons (n = 184) and treatment comparisons (n = 32) were also not included in their respective Hedge’s g calculations as the standard deviation for at least one of the conditions was zero. Therefore, the omnibus Hedge’s g effect sizes were calculated using only 91 of 293 (31%) IISCA condition comparisons and 22 of 73 (30%) treatment condition comparisons.

Additionally, the authors could only report on IISCAs and subsequent treatments that were published in peer-reviewed journal articles or dissertations/theses. As Sham and Smith (2014) reported, single-case design studies are also subject to publication bias in which studies with favorable results are more likely to be published than studies with less favorable or contradictory results. Therefore, it is possible that the effect sizes found for the effectiveness of the IISCA and its subsequent treatments in the current study are higher than those found in unpublished IISCA studies and clinical practice.

Future Directions

Future studies on the IISCA might consider continuing to expand the types of participants recruited, increase the ecological validity of the IISCA and its subsequent treatments, collect and report procedural and treatment integrity, and describe IISCA and treatment implementer training. As mentioned before, 76% of participants in the current review have a diagnosis of autism spectrum disorder. Other developmental disorders such as intellectual disabilities and ADHD are also common disorders seen among the participants of IISCA. Therefore, future studies on the IISCA may consider recruiting participants without autism or developmental disabilities. Instead, more research should be conducted on the IISCA and its effectiveness with the typically developing population (i.e., 13% of the current participant pool). They may also consider recruiting older participants (i.e., teenager-aged and adults), as 92% of IISCA participants have been below the age of 15.

Additionally, as 73% of SCAs have been conducted in a clinical setting (i.e., outpatient clinic or university-based clinic), future studies should seek to increase ecological validity by conducting more analyses in the participants’ natural environments (i.e., home, school, and community settings). While 15% of SCAs have been conducted in a school setting, only one participant had their IISCA conducted in a non-specialized school classroom. All other school-based IISCA analyses were conducted in a specialized school (i.e., a school designed for children with autism) or in a separate room (e.g., a therapy room) within their school. Similarly, based on the studies included in the current review, only one teacher (i.e., 0.3% of implementers) and nine parents (3%) have implemented an IISCA. Therefore, future studies should consider using more natural change agents (i.e., individuals in the participant’s natural environments) to implement the IISCAs to further extend the ecological validity of the assessment.

Furthermore, in the current review, 77% of IISCAs and 64% of treatment analyses did not report procedural integrity and treatment integrity data, respectively. Similarly, 84% of IISCAs and 87% of treatment analyses did not report how implementers were trained on assessment and treatment components. An additional 15% of IISCAs and 3% of treatment analyses only reported that implementers had previous training in functional analysis or conducting function-based interventions. Therefore, future IISCA researchers should also make it a priority to collect and report procedural and treatment integrity data and report implementer training methods during IISCAs and treatment analyses.