Eligibility assessment for special education services has traditionally been the largest job role engaged in by school psychologists nationwide (Bramlett et al., 2002). This finding continues to be reported in more recent surveys (Benson et al., 2019; Galanaugh, 2018; McNamara et al., 2019). Results of the 2015 membership survey of the National Association of School Psychologists indicate that although the primary role of school psychologists continues to be completing eligibility evaluations for IDEA services, the role of school psychologists in supporting the mental health needs of students is increasing (McNamara et al., 2019). Of those children receiving mental health services, 70–80% receive services through the schools (McNamara et al., 2019). More frequently than before, school psychologists are being called on to participate in treatment planning and implementation (Bahr et al., 2017). In their investigation into the roles and responsibilities of school psychologists across three Midwestern states (Missouri, Illinois, and Iowa), Bahr et al. (2017) found that school psychologists, at least in the states included in the survey, were spending more time on treatment planning and consultation than diagnostic assessment. This shift towards a more interventionist perspective is consistent with the expressed interests of practicing school psychologists in expanding their involvement in intervention and consultation (Sullivan et al., 2011).

The shifting role may have, at least in part, been influenced by the passage of PL 105-17 in 1997, which mandated that a functional behavior assessment be conducted in cases of interfering behavior in children eligible for special education services (Benson et al., 2019). The concept of functional behavior assessment is rooted in the theories and practices of applied behavior analysis (ABA) (Cooper et al., 2020). The completion of a functional behavior assessment, prior to developing behavioral treatment, is considered current best practice within both educational and psychological settings (Dufrene et al., 2017), and 75% of experienced school psychologists report that they have increased their use of behavioral assessment procedures over the course of their career (Shapiro & Heick, 2004). Although written into both federal and state regulation, the specific procedures to use when conducting a functional behavior assessment or developing a behavior intervention plan are not outlined (Johnson et al., 2018; Wilczynski et al., 2002). One would then think that school psychologists who regularly conduct functional behavior assessments would be guided by the literature; however, it has been found that the school psychology literature provides little information about the actual procedures that go into the functional behavior assessment process (Johnson et al., 2018).

Another trend in school psychology that should have increased the presence of applied behavior analysis (ABA) within the school psychology literature is the Response to Intervention (RTI) movement that started in the early 2000s and was included in the IDEA Improvement Act of 2004 (Fan et al., 2016). With an emphasis on prevention, RTI models address behavioral and learning needs in students through early intervention based on individualized instructional need. The focus is shifted away from traditional student assessment, based on a classification model, and toward a behavioral assessment based on an environmental model (Ardoin et al., 2016). The goal of RTI is to improve student learning and behavioral performance within the classroom, as much as possible, thereby avoiding the need for classification and self-contained instruction. This is accomplished through instructional modifications aligned with evidence-based instruction (Hicks et al., 2014), ongoing student assessment, and data-based decision making (Fan et al., 2016). With its focus on observable student targets, functional behavior assessment, objective and ongoing performance measurement, assessment and modifications of environmental stimuli, single-subject intervention designs, and evidence-based practice, the RTI model is seen as being generally based on an ABA framework (Ardoin et al., 2016).

Best practice procedures for behavioral assessment and treatment are rooted in the theory, philosophy, and techniques of applied behavior analysis (ABA) (Cooper et al., 2020). As evidenced by the FBA and RTI movements, in many respects, the fields of ABA and school psychology are complementary in that both seek to increase student learning of academics as well as encourage appropriate behavior in and beyond the classroom. Vollmer and Northup (1997) outlined four features of ABA that make it a useful discipline for student assessment and school intervention: 1) an emphasis on analysis, 2) an emphasis on repeated measures of individual behavior, 3) an emphasis on observable behavior and environmental events, and 4) a reliance on principles of behavior to support behavior change. Although acquiring skills in ABA would improve the ability of school psychologists to provide direct and indirect assessment and treatment services, the reliance on empirically based ABA principles and procedures by school psychologists is not often found (Runyon et al., 2018) or is inconsistent. An example of this inconsistency can be found in the use of various functional behavior assessment procedures. In their survey of 1317 practicing school psychologists, Benson et al. (2019) found that 61.7% of respondents reported using direct ABC data when conducting functional behavior assessments, 53% reported conducting direct interviews with teachers, 46% direct interviews with parents, and 50.4% collecting direct observation frequency data when developing behavior interventions. Johnson et al. (2018), in their survey of 199 school psychologists, found that when conducting functional behavior assessments, 91% reported using direct ABC data, 89% reported using direct frequency data, 70% reported using direct time sampling data, and 59% reported using permanent products. Despite these encouraging data, broad-based, indirect and/or subjective behavior rating scales are still the most widely used method of assessment for the treatment of problem behavior employed by school psychologists (Benson et al., 2019). These indirect methods are often relied on because of their ease of use and because many school psychologists lack expertise in more empirically supported, ABA assessment procedures (Dufrene et al., 2017). Although we are certainly not suggesting that school psychologists need to obtain certification in behavior analysis in order to competently administer ABA assessments and interventions, it is interesting to note that 1% of school psychologists surveyed were already dually certified at the time of the most recent NASP membership survey (Walcott et al., 2018). One hypothesis on the lower than expected use of ABA procedures by school psychologists is a potential lack of self-efficacy in the practice of ABA on the part of school psychologists (Runyon et al., 2018). As school psychologists receive more training in ABA, increases in self efficacy may result in increased use of ABA procedures in practice (Runyon et al., 2018).

Scholarship, through published literature, forms the foundation for a discipline and its clinical professions (Liu & Oakland, 2016). As with other fields, research and literature published in professional journals exert the most influence on school psychologists, both through direct review of the literature and indirectly through books, professional standards, and the development of policies based on the literature (Villarreal et al., 2013). Unfortunately, it has been found that psychologists, including school psychologists, infrequently read professional journals (Shaw, 2016). This results in a research to practice gap with regard to mental health services provided within schools. For example, Hicks et al. (2014) reported fewer than expected nationally certified school psychologists reporting familiarity with behavioral evidence-based interventions and promising practices. Due to the practice of school psychology adopting more behavioral procedures, Reed (2008) investigated the scientific translation occurring between the research in ABA and school psychology. The author analyzed the frequency of shared citations between two ABA journals (Journal of Applied Behavior Analysis and Journal of the Experimental Analysis of Behavior) and four school psychology journals (School Psychology Review, School Psychology Quarterly, Psychology in the Schools, and Journal of School Psychology) in volumes published in 2006. Minimal overlap was found. No shared citations were noted between any of the school psychology journals and Journal of Experimental Analysis of Behavior. Shared citations were only found between Journal of Applied Behavior Analysis and School Psychology Review, and only 43 were noted. If research is suggesting that the principles and procedures of ABA are vital to the contemporary practice of school psychology (Runyon et al., 2018), it is logical that the two fields should begin to engage in translational research in order to reduce the research to practice gap. In addition, school psychologists are often more interested in reading practical research than in theoretical research (Shaw, 2016). Research describing how to implement behavioral treatments is highly useful to school psychologists in practice (Shaw, 2016) and is an important part of establishing the evidence-base of interventions to be used in schools. By making the literature more useful and attractive to school psychologists, it is here that ABA studies within the school psychology literature may be able to lessen the research to practice gap with regard to school psychology and the implementation of behavioral evidence-based practices.

Purpose of the Present Study

The purpose of the present study was to investigate the frequency of articles specifically discussing the theories and practices of ABA published in four school psychology journals, between the years of 2000 and 2020. Results will identify whether there is a deficiency of ABA presence in the current school psychology literature. As discussed earlier, studies consistent with the theories and procedures of ABA may be able to lessen the research to practice gap with regard to school psychology and evidence-based practice. This is especially important in a climate where the prevalence of problem behavior in schools is increasing in both special education and general educations populations (Runyon et al., 2018).

Method

Journal Selection

Four school psychology journals were selected for review. Selected journals, along with their current Impact Factor Scores and CiteScores, included Journal of School Psychology [JSP] (IF = 2.981, CiteScore = 4.7), School Psychology International [SPI] (IF = 1.431, CiteScore = 2.7), School Psychology, previously titled School Psychology Quarterly [SP/SPQ] (IF = 2.158, CiteScore = 3.12), and School Psychology Review [SPR] (IF = 1.920, CiteScore = 3.2). Liu and Oakland (2016) identified these journals as falling within the top five of school psychology publications, with JSP, SPR, and SP/SPQ having the highest frequency of citations in the Web of Science Database compared to other school psychology-related sources. Despite a lower ranking, SPI was included due to its international focus. In addition, three of the journals are publications of national professional organizations representing the field of school psychology; specifically SP/SPQ is a publication of the American Psychological Association Division 16 (school psychology), SPR is a publication of the National Association of School Psychologists, and JSP is a publication of the Society for the Study of School Psychology. These four journals, therefore, are seen as representing and reflecting the broad state of school psychology at this time. All article citations published within the selected journals, between the years of 2000 and 2020, were included in the initial review. Articles that were not original scholarly papers were subsequently removed from further analysis. This included book reviews, obituaries, conference summaries, award announcements, etc.

Search Method

The initial article search was conducted in August of 2020, by the first author, using the multi-database search engine available through the Seton Hall University library system https://library.shu.edu/library. The Seton Hall University library system has the capacity of searching through 474 databases simultaneously. Some of the more frequently used databases available through the system include Academic Search Complete, APA PsycARTICLES (EBSCO Publishing), APA PsycINFO (EBSCO Publishing), The Cochrane Library, Education Database 1988-current (ProQuest Central), ERIC (ProQuest), Google Scholar, PubMed, SAGE Journals Online, ScienceDirect, Springer Journals, Taylor & Francis Online, and Wiley Online Library. For the current search, each journal title was entered in quotation marks and the search was limited to the term being found in the Journal/Source of the citation. Results were then limited to the years 2000-2020.

Once obtained through the initial search, results were exported to Zotero 5.0.88 for easier storage and to automatically identify and remove duplicates. Citations in JSP were gathered first. Due to an extensive number of duplicate citations, the remaining three searches were conducted with the database limited to Academic Search Complete. Results were then exported to a Microsoft 365® Excel spreadsheet for ease of analysis. Authors were assigned a journal to review and were sent the corresponding Excel spreadsheet via email. Authors first conducted a final check for duplicate citations, which were then removed by hand.

The remaining articles were then reviewed according to the criteria for inclusion outlined below. Reviews were initially conducted by reading the title and abstract of each citation, which were included in the Excel spreadsheet as exported from Zotero. If an author was unable to make a determination of inclusion based solely on information provided in the title and abstract, the full text of the article was retrieved and reviewed. Twenty percent of citations were subjected to inter-rater reliability review by having the current authors review each other’s citation lists. Articles that met inclusion criteria were then analyzed as to whether or not they reflected the experimental analysis of a behavioral intervention and, if so, were the behaviors of interest behavior excesses (target behaviors to decrease), behavior deficits (target behaviors to increase), or both.

Criteria for Inclusion

Once the citations were placed in Excel spreadsheets, the titles and abstracts of each citation were subjected to the following inclusion criteria:

  1. 1.

    The article must be an original work of scholarly research or discussion (i.e., not tables of contents, obituaries, book reviews, retractions, call for papers, etc.).

  2. 2.

    The article must have a primary focus of a treatment, concept, or discussion topic consistent with theories of operant or respondent conditioning, or discussing a topic directly related to their effective implementation (i.e., functional behavior assessment procedures, teacher training on behavioral treatments, etc.)

Operational Definitions

Cooper et al. (2020) was used as a reference for the operational definitions of operant and respondent conditioning.

Operant conditioning – The basic process by which operant learning occurs; consequences (stimulus changes immediately following responses) result in an increased (reinforcement) or decreased (punishment) frequency of the same type of behavior under similar motivational and environmental conditions in the future (Cooper et al., 2020 p. 796).

Respondent conditioning – A stimulus-stimulus pairing procedure in which a neutral stimulus (NS) is presented with an unconditioned stimulus (US) until the neutral stimulus becomes a conditioned stimulus that elicits the conditioned response (Cooper et al., 2020 p. 798).

Results

The reader is referred to Fig. 1 for a visual display of the search results depicted through a modified PRISMA 2009 Flow Chart. The initial search resulted in the identification of 7097 citations that were published in the selected journals between the years of 2000 and 2020. After duplicates were removed, a total of 2918 unique citations were obtained (796 from JSP, 776 from SPI, 590 from SP/SPQ, and 756 from SPR). These 2918 citations were reviewed according to the two inclusion criteria; 153 were subsequently removed based on not meeting the first criterion and an additional 2612 were removed based on not meeting the second criterion. This resulted in a final list of 153 articles that were determined to be reflective of ABA theory and practice.

Fig. 1
figure 1

Modified Prisma 2009 flowchart displaying detailed search results

Full size image

Inter-rater reliability (IRR) data were collected on 20% of unique citations within each journal (n=582). IRR was calculated by taking the percent of reviewer decisions that matched out of the total (match + no match). A total IRR of 93.3% was obtained with a breakdown of 90% for JSP, 97% for SPI, 91% for SP/SPQ, and 94% for SPR. These percentages reflect a high level of reliability in the results based on the search procedures and operational definitions of the inclusion criteria used in the current analysis.

Of the 2765 articles that met the first inclusion criterion only 153 (5.5%) met the second inclusion criterion, indicating that they had a primary focus consistent with ABA theory and practice. JSP showed the highest frequency of ABA-consistent articles (66/723 or 9.1%) followed by SPR (53/727 or 7.2%), SP/SPQ (18/560 or 3.2%), and SPI (16/755 or 2.1%). Data are represented graphically in Fig. 2.

Fig. 2
figure 2

Percent of citations with a primary focus consistent with ABA, published between 2000 and 2020

Full size image

Figure 3 displays the number of ABA consistent articles that were experimental analyses of a behavioral intervention collapsed across all four journals. Out of the 153 articles that were found to reflect ABA theory or practice, 113 (74%) were experimental investigations. The remaining 40 citations consisted of articles such as systematic reviews, meta-analyses, theory discussions, and practice descriptions. Out of the 113 experimental investigations, 59 (52%) focused on the treatment of behavioral deficits, 32 (28%) on the treatment of behavioral excesses, and 22 (19%) on the treatment of both deficits and excesses.

Fig. 3
figure 3

Number of citations that were experimental analyses of interventions along with their primary focus

Full size image

Discussion

Overall results indicated a low frequency of ABA articles in the school psychology literature over the past 20 years. Out of 2765 original research citations published within four of the top journals in school psychology, only 153 (5.5%) were found to have content with a primary focus reflective of ABA philosophy, theory, or practice. At 9.2%, JSP was found to have the highest percentage of ABA-consistent articles and at 2.1%, SPI was found to have the lowest. Given the shift in school psychology practice towards a more treatment focused, behavioral perspective, this level of ABA presence in the school psychology literature seems to be a missed opportunity to reduce the research to practice gap found in the field. There is also a concern that this low presence of literature will result in training gaps related to the provision of mandated services that are based within ABA (i.e., functional behavior assessment, response to intervention, use of evidence-based interventions, etc.). Although only 113 articles were found to be experimental analyses of behavioral interventions, it was interesting to find a focus both on target behaviors to increase and targets behaviors to decrease. This indicates that practices consistent with ABA are being employed by school psychologists not only to decrease problem behavior in students but to increase appropriate behavior and academic achievement.

If it is true that the field of school psychology is gradually shifting from a primarily evaluative discipline to one having a larger role in evidence-based behavioral treatment, then it is logical that our literature starts to reflect this change. The low frequency of ABA content in the school psychology literature, as indicated by the current investigation, could set the stage for incorrectly conducted functional assessments, recommendations for contraindicated treatments for problem behavior, less than optimal student academic progress, and promotion of outdated and non-empirically supported behavioral procedures.

The current analysis and conclusions are qualified by three potential limitations. First, only four school psychology journals were chosen for review. Not only are there additional journals in the field of school psychology, but it is common for relevant studies to cross disciplines and be published in journals of separate but related fields (i.e., counseling psychology, education, etc.). Second, it is possible that articles were behavior analytic in nature, however, being tailored to school psychologists and not behavior analysts, did not use terminology common in ABA within the title or abstract. Therefore, it is possible that some articles were not identified through the search method employed. The third limitation is directed towards the implications of the current results. Since school psychologists can review literature published directly within behavior analytic journals (i.e., Journal of Applied Behavior Analysis, Behavior Analysis in Practice, etc.), the negative impact of the current results may be somewhat less than predicted. Despite the potential limitations, if it is true that many school psychologists are reporting an increased use of behavioral assessment and treatment procedures, as well as a lack of training in those procedures, then having a literature base, directly in the field, with only five percent of articles addressing the need is not ideal.

Future Areas to Explore

Increasing the presence of ABA-focused research in the school psychology literature can have a significant impact on the clinical services provided by contemporary school psychologists. One thing that would be interesting to explore is the relationship between an increase in functional assessment research and methods used by practicing school psychologists. In their investigation of the use of various functional assessment methods reported by 140 school psychologists and 123 special educators, O’Neill et al. (2015) found that special educators reported a significantly higher frequency of the use of direct observation and functional analysis procedures as compared to school psychologists as well as a higher opinion of the value of direct student observation when assessing function of behavior. School psychologists reported a greater willingness to use indirect methods of functional assessment, such as rating scales and interviews. In narrative responses, school psychologists often reported that direct observation methods were too time-consuming. This is unfortunate in light of decades of research indicating that direct observation of behavior yields more confident results of function than indirect methods (Cooper et al., 2020). Regarding the realistic issue of needing to keep assessments to a practical duration, behavioral technology has addressed the problem through the development of brief functional analysis procedures (Tincani et al., 1999), which were found to yield the same results as traditional functional analyses but in a 20% shorter period of time. The development of latency-based functional analyses (Thomason-Sassi et al., 2011) and in-classroom trial-based analyses (Bloom et al., 2011) has also increased the practicality of evidence-based functional analysis procedures for use by the school practitioner. Most recently, work by Gregory Hanley and colleagues, has modified the traditional view of functional analysis to include the concept of combined or “synthesized” functions and developing brief test-control functional analyses through behavioral interviews with caregivers (Hanley et al., 2014). By using these “Interview-Informed Synthesized Contingency Analyses,” Jessel et al. (2016) were able to analyze the function of problem behavior across 30 cases with an average duration of only 25 min per case. The practice of school psychology could benefit from studies such as these within our own literature base.

Another area to be further explored by school psychologists is the use of behavior analytic principles and procedures for the advancement of skill acquisition in students, not just to decrease problem behavior. According to B.F. Skinner, all skill acquisition is based on behavioral principles (Vargas, 2020). Teaching, in its most basic form, is the simple arrangement of contingencies in the classroom to produce desired behavior change in students (Vargas, 2020). Through the use of empirically supported teaching technologies, such as Precision Teaching, Direct Instruction, and Fluency Training, which are based on the principles of behavior analysis, we can significantly improve student learning (Binder & Watkins, 2013). Despite a seminal article published in 1971 in the Journal of School Psychology titled Precision teaching: A tool for the school psychologist and teacher (Alper & White, 1971), it is unfortunate that these teaching methodologies, designed for skill acquisition, are underused in schools today (Binder & Watkins, 2013). Student performance may be greatly improved with an increased awareness of these teaching procedures by school psychologists.

Another area to explore is how much the practices of the field may be influenced by test developers and publishing companies who stand to make a profit when school psychologists purchase rating scales and standardized interviews in place of relying on direct observation data when assessing student behavior. Direct observation in natural settings is a more reliable and accurate method for assessing the maintaining variables of student behavior (Cooper et al., 2020); however, their use does not result in profit for test developers and publishers. Given their often-oversized caseload, school psychologists realistically need to be concerned with the efficiency and practicality of the procedures that they use. The ease and speed of published rating scales and standardized interviews makes them highly attractive. A related concern that may increase the use of rating scales and standardized interviews by school psychologists is the using of assessment reports for placement decisions and legal proceedings. In an effort to be viewed as unbiased and to lessen liability risk for themselves and school districts, school psychologists may prefer to use methods that are standardized and produce quantitative scores (i.e., BASC-3, MAS, etc.) rather than methods that are more open ended (i.e., ABC data) or interpreted through visual display of in vivo data without permanent product (i.e., functional analysis). Unfortunately, these indirect methods are often more subjective than well executed direct observation methods and therefore are the opposite of what would be empirically supported when it comes to ensuring quality student outcome (Cooper et al., 2020).

In order to address these issues, as well as other areas of contemporary school psychology practice, a call for more ABA-focused research, published directly in the school psychology literature, is recommended.