Introduction

According to the National Assessment of Educational Progress (NAEP 2015), only 40% of fourth-grade students and 33% of eighth-grade students are at or above the levels of proficiency established by NAEP in math. Underdeveloped skills in mathematics can lead to lifelong difficulties and extend into other facets of adulthood. Research suggests mathematical competence accounts for variance in employment, income, and work productivity (National Mathematics Advisory Panel 2008). Students who are competent in mathematics are also more likely to seek careers in science, technology, engineering, and mathematical fields, which are associated with higher pay (National Research Council 2001). Given the extent to which success in mathematics can influence the future lives of students, it is critical to promote mastery of the basic math skills that are necessary for the learning of advanced mathematical concepts (Shapiro 2011). A necessary aspect of student numeracy development is multi-digit computation. Multi-digit multiplication is one of the main standards for fourth-grade students under Common Core (Common Core State Standards Initiative 2017). However, before student acquisition of these more advanced skills can be addressed an understanding of the sources of student underachievement is required.

Skill- and Performance-Based Deficits

The instructional hierarchy (IH) is a behavioral means for categorizing student academic performance in a manner that can guide differentiated instruction as students progress through various skills (Haring 1978). According to the IH, students first acquire accuracy with a given skill before developing fluency with that skill. Once students can perform a skill with accuracy and fluency, the final two stages of the IH are generalization and adaptation. However, this is not to say that the levels of the IH are impermeable and that students are only ever on one level at a time with a given skill. On the contrary, it is possible for students to be developing fluency as they are acquiring accuracy with a skill.

Students with skill-based deficits generally lack the knowledge to perform the skill with accuracy and or fluency. According to the IH, instruction targeting accurate responding should include modeling of how to respond to stimuli, immediate feedback regarding response accuracy, and reinforcement for accurate responding (Ardoin and Daly 2007; Haring 1978). Once accuracy is established, instruction should shift to target fluency to enable students to complete associated tasks quickly and with little effort. Fluency instruction should include repeated practice with reinforcement for accurate and fluent responding. To maximize the effectiveness of fluency instruction, students’ rate of responding to stimuli should not be encumbered by the individuals implementing the intervention or the steps of the intervention itself (Daly et al. 1997; Haring 1978).

In contrast to skill-based deficits, students with performance-based deficits can perform the target skill with accuracy and fluency but lack the motivation to complete the skill with accuracy or fluency. Performance-based deficits can generally be remediated by providing reinforcement for desired responding (Duhon et al. 2004). However, performance-based deficits are sometimes accompanied by skill-based deficits in which case a combination of skill- and performance-based intervention components are necessary (Daly et al. 1997; Skinner 1998).

Brief Experimental Analysis (BEA)

Brief experimental analysis (BEA) is a methodology for identifying what intervention component(s) is likely to be most effective for remediating a student’s deficit (Martens and Gertz 2009; McComas and Burns 2009). The procedures for BEA of academic behavior generally involve rapidly implementing multiple interventions and observing the effects of each intervention on student performance (Daly et al. 1997). Data from BEAs are plotted in single-case design fashion, and visual analyses are conducted to determine which intervention produces the greatest improvement in student academic performance. However, it is important to not only consider which intervention results in the greatest improvement but also to consider the required effort necessary for implementation of each intervention. Interventions should aim to be minimally intensive for the given situation and are viewed as efficient when the lowest possible intensity is paired with maximal effectiveness (Daly et al. 1996, 2002; Harding et al. 1994).

To date, the majority of research examining BEA of academic performance inspects the utility of BEA for improving students’ oral reading fluency (Burns and Wagner 2008; Daly et al. 1997, 1999). In large part, these studies examined individual students’ responses to three interventions: (a) listening passage preview with repeated readings, (b) repeated readings, and (c) contingent reward (CR). These interventions target the accuracy and fluency stages of the IH, fluency only, and performance-based deficits, respectively.

Although few in number, researchers have also examined the potential benefits of using BEA in the identification of effective math interventions (Codding et al. 2009; Gilbertson et al. 2008; Reisener et al. 2016). Initial studies systematically tested an array of interventions but focused on one type of deficit or stage of the IH, such as accuracy. Testing interventions that address only one type of deficit prevents the testing of alternative hypotheses related to other skill- or performance-based deficits. For instance, Hendrickson et al. (1996) utilized BEA to compare the effects of two accuracy-based interventions [cover, copy, compare (CCC) and constant time delay (CTD)] with a student for whom they believed had a skill-based deficit. CCC consists of students viewing a model of the problem, covering the model, answering the problem, and then comparing their answer to the answer in the model (Skinner et al. 1989). CTD procedures involve the instructor first modeling the problem to the student and then providing the student with the opportunity to provide a response. If the student takes longer than 5 s to respond or provides an incorrect response, the instructor provides the correct answer (Koscinski and Gast 1993). Similarly to Hendrickson et al. (1996), only accuracy-based interventions (CCC, taped problems, Math to Mastery) were examined by Mong and Mong (2012) when using BEA to identify the most effective math intervention for three elementary students. Use of BEAs that include interventions targeting only one type of skill deficit make the assumption that the examiner already knows a student’s instructional needs.

Fortunately, consistent with the BEA literature on oral reading fluency (Daly et al. 1999), a few researchers have examined both skill- and performance-based deficits in their illustration of the effectiveness of BEA with mathematics (Carson and Eckert 2003; Reisener et al. 2016). For example, Carson and Eckert (2003) evaluated the effects of utilizing BEA to identify intervention components that would be beneficial for three students with performance-based deficits in math. CR, goal setting, performance feedback, and timed sprints were included as interventions. The authors compared students’ response to the BEA-selected intervention to a student-selected intervention in an effort to evaluate the effect of student choice. Although results did not indicate choice to improve performance, the timed sprints intervention selected through BEA was effective for all three students. Reisener et al. (2016) employed a BEA to evaluate whether CR, CCC, or CTD best met the instructional needs of eight elementary aged students. An extended analysis comparing the least and most effective interventions identified by the BEA indicated that the BEA procedures successfully identified the most effective intervention for 6 of the 8 students. Unfortunately, analyses excluded an intervention targeting fluency and thus failed to examine whether the students may have benefitted from such an intervention or a combination of intervention components.

Although schools may occasionally believe they know what intervention would best meet a student’s needs, BEAs should be employed that test their hypotheses and thus include interventions that target all potential types of deficits (accuracy, fluency, and performance-based deficits). To the author’s knowledge, although several BEA studies on mathematics have examined interventions that target accuracy, only one study has examined a fluency-based intervention (Carson and Eckert 2003) and none have included an intervention for each deficit or stage of the IH within a single study. The current study was conducted in an effort to fill that gap in the literature. When viewing BEA as a methodology for assessing and categorizing student performance, it would follow that if effective intervention is provided, then a student’s need may change rapidly as they progress through the IH. Therefore, a need for re-evaluating student performance exists. It is hypothesized that a follow-up or second BEA could provide important information regarding student skill development and whether a point is reached at which an intervention not identified through the initial BEA as the most effective could become the most effective. For instance, if an accuracy-based intervention is identified in the initial BEA as producing the greatest effects on behavior, and an accuracy-based intervention is then implemented, the subsequent BEA should identify a fluency intervention as most effective. Such data would provide evidence of the validity BEA for identifying effective interventions and being sensitive to students’ instructional needs.

Consistent with the BEA oral reading fluency and math research the interventions selected for the current study included interventions that targeted accuracy and fluency [CCC plus repeated practice (CCC+RP)], fluency only [repeated practice (RP)] and a performance-based intervention [contingent reinforcement (CR)]. CCC requires a student to view a model of the problem, cover the model, answer the problem, and then compare the student’s answer to the answer in the model (Skinner et al. 1989). Given these components, it has regularly been used within the BEA literature to target accuracy deficits and extensive research provides evidence of its effectiveness for improving math proficiency for students with and without disabilities (Codding et al. 2007; Joseph et al. 2012; Poncy et al. 2007). Not surprisingly, research indicates that CCC is most effective for students in the frustrational range according to their digits correct per minute (DCPM), suggesting that they need an intervention targeting accuracy. However, the effectiveness of CCC wanes as students need an intervention targeting response fluency (Burns et al. 2010; Codding et al. 2007). As previously mentioned, students who are gaining accuracy with a skill can simultaneously develop fluency with that skill and might benefit from an intervention that targets both of these IH stages. Thus, drawing from the BEA oral reading fluency literature in which researchers have frequently found students respond best to interventions that provide students with both modeling of accurate responding and multiple opportunities to practice responding, the current study examined the effects of a CCC+RP intervention on students’ performance (Daly et al. 1999, 2002).

Congruent with the BEA oral reading fluency literature, the current study adapted repeated reading procedures to target mathematics skills. Although prior researchers have employed CTD as their intervention targeting fluency, the associated procedures require students to provide the answer to a math problem in a designated amount of time before the interventionist supplies the answer. Due to the modeling component of CTD as well as students’ inability to advance to the next problem on their own, it might be more appropriate to designate CTD as an accuracy-based intervention. The RP intervention tested in the current study and employed both in combination with CCC and alone draws from IH research demonstrating the effectiveness of providing students with multiple opportunities to practice basic math skills through drill and practice (Burns et al. 2010). The current study extends that literature by implementing the selected interventions with an alternative method for conducting multi-digit multiplication, the area model.

When utilizing the area model, individuals break the two numbers being multiplied together into their tens and ones values (e.g., 52 × 48 into 50, 2 and 40, 8). Both values that constitute a single number are then multiplied with the two values of the other number. The four products (e.g., 50 × 40, 50 × 8, 2 × 40, and 2 × 8) are then added together to obtain the final answer (e.g., Tillema 2009). An example of a two-digit by two-digit multiplication problem solved utilizing the area model is provided in “Appendix”. To the author’s knowledge, no research studies have been conducted with the area model, especially any studies with the interventions included in the current study.

Purpose

The current study sought to extend the literature of BEA with mathematics in multiple ways. First, BEA oral reading fluency procedures outlined by Daly and colleagues (Daly et al. 1999, 2002) were adapted to evaluate math interventions that target each of accuracy, fluency, and performance-based deficits. In this manner, the included interventions addressed the acquisition and fluency stages of the IH as well as any students who were capable but unwilling to complete the task (Daly et al. 1997; Haring 1978). Second, previous BEA studies focused on basic mathematical operations such as single-digit addition and subtraction or multiplication facts (Codding et al. 2009; Mong and Mong 2012; Reisener et al. 2016). A natural progression in the study of BEA was to test BEA with increasingly advanced mathematics. Given the inclusion of CCC+RP and RP as interventions, the current study also investigated whether these interventions could improve students’ accuracy and fluency in completing two-digit by two-digit multiplication when using the area model method. To measure students’ accuracy and fluency the current study sought to extend the literature by utilizing two dependent variables in combination, DCPM and digits incorrect per minute (DIPM).

A final purpose of the current study was to examine the potential benefits of conducting follow-up BEAs following implementation of an intervention identified as most effective. To date within the BEA research interventions identified through the BEA have been implemented following the BEA as a means of demonstrating that the BEA in fact identified an effective intervention (Mong and Mong 2012; Reisener et al. 2016). However, the current study sought to demonstrate that if an effective intervention is identified through a BEA and that intervention addresses the student’s instructional needs, a follow-up BEA should identify an intervention that addresses the next level of the IH.

Method

Participants and Setting

Teachers from a public middle school in a southeastern state were asked to identify students who exhibited inaccurate responding with two-digit by two-digit multiplication as potential participants for this study. Four students without a special education diagnosis but whom were receiving tier 2 response to intervention mathematics services were identified. Don and Jose were 13-year-old, Hispanic males in seventh-grade, and Evan and Ben were 12-year-old, Hispanic males in sixth grade. All study procedures were approved by the authors’ university-based institutional review board and each student’s parent provided consent for their child to participate. Intervention sessions were conducted in an unoccupied classroom directly adjacent to students’ classrooms and lasted approximately 2 to 15 min. Students participated in two consecutive sessions per day each school day they were present over the course of 2 months.

Materials

Math Worksheets

Two-digit by two-digit multiplication with and without regrouping math worksheets were utilized during baseline, BEA, and intervention phases. Each worksheet contained four problems with the following qualities: (a) at least one number greater than 12 to avoid simple math facts (i.e., 12 × 11) and (b) both numbers being greater than 10. An online math worksheet generator (i.e., interventioncentral.org) was utilized to develop the math problems, which were then inserted onto math worksheets and configured for the area model in all intervention conditions. The math worksheets and generalization probes were standardized according to these qualities for all students.

Generalization Probes

During the intervention phase, generalization probes were administered after every third session. Generalization probes were formatted identically to the math worksheets but contained different problems from the ones students had completed during the preceding intervention session.

Dependent Variables

The primary dependent measure was DCPM and was calculated using curriculum-based measurement scoring procedures detailed by Shinn (1989). Every digit that was correct and in the proper column was scored as correct. In adaptation for the area model, only digits that participants calculated were scored. Therefore, the four products that were the result of multiplying across the boxes of the area model and the sum of those four products were scored. An example of this scoring procedure is provided in “Appendix”. DCPM was calculated by dividing the number of digits correct by the number of seconds worked and then multiplying by 60. The second dependent measure employed was digits incorrect per minute (DIPM) which was calculated in the same manner as DCPM, except the number of incorrect digits was tallied. Participants’ performance with two-digit by two-digit multiplication was categorized as frustrational (0–19 DCPM), instructional (20–39 DCPM), or mastery (40 + DCPM) by extrapolating from Burns et al. (2006), Deno and Mirkin (1977) and Shapiro (2011).

Procedures

Experimental Design

For each pair of students, a multiple baseline design was used to aid in evaluating the effects of the selected intervention. After the collection of baseline data was completed for the first student, a BEA was conducted with the three interventions. Consistent with past BEA research (e.g., Daly et al. 2002), the interventions were implemented from least to most intensive (CR, RP, CCC+RP). Ordering the interventions in this fashion allowed for avoiding potential generalization effects as students who were in the acquisition stage of the IH would not receive a modeling component until the final intervention in the BEA. The most and least effective interventions were then implemented an additional time in an alternating fashion to verify the most effective intervention. If two interventions were suggested to be equally effective, the least intensive of the two was selected in order to utilize the most efficient intervention. Once visual analyses indicated an improvement in responding, BEA procedures were implemented with the second student. Then the second student was provided with the intervention found to be most effective for him.

Baseline

Students were administered a single math worksheet. Although classroom instruction was being provided on skills that required students to know two-digit by two-digit multiplication, direct instruction was not being provided on this skill by any teacher during baseline or any other condition of this study.

Contingent Reward (CR)

Any improved performance in DCPM or DIPM accompanied by stable or improved performance in the other variable was established as the criteria to receive a reward in the CR condition. Each session, students were provided a worksheet to complete and instructed that any improvement in their performance from the previous session would result in them receiving a reward. The criteria of “any improvement” were selected due to the 1.0 DCPM ambitious weekly growth estimate for sixth-grade students outlined by Fuchs and Fuchs (1993). Since sessions were conducted with students every day of the week, essentially any increase in DCPM, along with stable or improved DIPM, would qualify to meet this weekly growth estimate. When students met the criterion, they were allowed to choose a tangible item from among a variety of items (e.g., candy, bouncy balls, etc.) regularly used as rewards for students in school settings.

Repeated Practice (RP)

The RP condition required students to complete a set of 4 two-digit by two-digit multiplication problems three times. Sets of problems were completely random each session and followed the guidelines listed for math worksheets above. Each problem was on a separate half sheet of paper, and students were provided the sheets of paper for all of the problems in the set. After the student completed all of the problems in the set, the student was given the same set of problems to complete a second time and then a third time. Upon completing the set of problems the third and final time, the administrator provided the student with corrective feedback on each problem if necessary. DCPM and DIPM were calculated on the final set of problems.

Repeated Practice Plus Contingent Reward (RP+)

One session of RP+ was conducted in the study during the BEA for Don. RP+ was identical to RP, but the opportunity for a reward was available if student performance on the third time solving problems met the CR criteria describe above. This condition was created specifically for Don due to his higher level of performance and a hypothesis that the RP+ condition might be most effective for him.

Cover Copy Compare Plus Repeated Practice (CCC+RP)

During the CCC+RP condition, students completed two-digit by two-digit multiplication problems with a worked example of the area model method to solve the problem. To begin the first session of CCC+RP, the author modeled the very first problem using CCC and thinking aloud with a script before the participant completed the remaining problems in the same fashion. The experimenter did not model the first problem prior to the remaining sessions, but rather participants were simply prompted to begin working on the problems using CCC. The process of CCC required students to look at the modeled problem, cover the model, copy the answer to the problem, and compare their answer to the model (Skinner et al. 1989). Due to response complexity concerns for students attempting to use CCC with the area model, modeled CCC problems were broken down into three sections: separating the two-digit numbers into their positions on the area model box, conducting the four multiplication problems, and adding the four products to obtain the final answer. Students worked through the method of CCC described above for each of these three sections. As the participants worked through the problems with CCC, they also had a script they were instructed to read which walked them through the problems. When a student correctly solved a problem, they were instructed to proceed to the next problem. When a student incorrectly solved a problem, they were given immediate corrective feedback and instructed to attempt the same problem again. After completing the CCC problems, RP was provided as students worked the exact same problems an additional two times. The administration of these two RP sets followed the procedures of the RP condition with students being timed on the final attempt of the problem set. However, students received corrective feedback and were required to attempt an incorrectly solved problem again after both sets of problems instead of only after the final attempt as in the RP condition.

Intervention Implementation and Second BEA

After an intervention was identified as most effective through the BEA, the identified intervention was implemented in isolation as a means of further demonstrating intervention effectiveness which is consistent with past research (e.g., Reisener et al. 2016). For students who responded best to the CCC+RP condition, a second BEA was conducted after the intervention was implemented in isolation and the student’s data indicated he had achieved accuracy (i.e., DCPM stabilized with near zero DIPM). The purpose of the second BEA was to demonstrate the sensitivity of BEA to changes in students’ instructional needs. Evidence that after. In the event of a student receiving a second BEA due to improved accuracy from the CCC+RP condition, implementation of the intervention identified as most effective during the second BEA was conducted in isolation.

Data Analysis

Researchers returned students to a baseline phase, if necessary, to provide a comparison with their paired student’s initial intervention phase so that such a baseline and intervention comparison existed across all students. A return to baseline was only necessary for Jose to provide a baseline comparison against Don’s initial intervention phase. DCPM and DIPM were considered together for all data-based decisions. An intervention was identified as most effective when it resulted in the greatest improvements in both DCPM and DIPM as compared to the other interventions. In cases where one dependent variable changed in a clinically positive direction and the other a negative, the two were weighed against each other for a net change when compared to the effects of other interventions.

Procedural Fidelity and Interscorer Agreement

A procedural checklist was developed for each phase (baseline, BEA, and intervention) and used in the assessment of procedural fidelity. Procedural fidelity was defined as the percentage of steps completed and was assessed through audio recordings of sessions. Undergraduate research assistants listened to 30% of the recordings. While listening to the recordings, the research assistants indicated whether or not each step was conducted as specified by the procedural checklist (e.g., read instructions correctly to student). Procedural fidelity was 100% for the current study.

Interscorer agreement was calculated for 30% of the worksheets completed by the participants during baseline, BEA, and intervention as well as the generalization probes. Agreement was calculated by dividing the number of agreed upon digits by the total digits scored and multiplying by 100. Scoring was done by the author and undergraduate research students on the final math worksheets students completed each day of intervention. Interscorer agreement for the current study was 99.88% (range: 97.36–100%).

Results

Visual analysis was used to evaluate the data from all phases of the study. Intervention effectiveness was determined based on the change in DCPM and DIPM from baseline. Results are shown in Fig. 1 (Jose and Don) and Fig. 2 (Evan and Ben). Additionally, Table 1 provides a summary of students’ performance.

Fig. 1
figure 1

Digits correct per minute and digits incorrect per minute under each condition for Jose (top panel) and Don (bottom panel). Generalization probe data are represented by the white markers. CR contingent reward, RP repeated practice, RP+ repeated practice with contingent reward, CCC+RP cover copy compare with repeated practice. Note: the DIPM for Jose in session 7 was 103 but is displayed as 80 for graphical purposes

Full size image
Fig. 2
figure 2

Digits correct per minute and digits incorrect per minute under each condition for Evan (top panel) and Ben (bottom panel). Generalization probe data are represented by the white markers. CR contingent reward, RP repeated practice, CCC+RP cover copy compare with repeated practice. Diagonal lines represent the break between the BEAs

Full size image
Table 1 Summary of participant performance
Full size table

Jose

Baseline

After completing five baseline worksheets, Jose’s performance suggested his two-digit by two-digit multiplication skills were in the frustrational range with an average of 13.5 DCPM (M DIPM = 40.39). His DIPM was highly variable and displayed an increasing trend, as he worked more quickly with each successive session.

BEA

Aligned with the order of least to most intensive, CR was administered first and as compared to baseline resulted in an increase in DCPM (26.67) but also an increase in DIPM (103.33), whereas RP resulted in a slight decrease in DCPM (9.72) and an increase in DIPM (51.89). CCC+RP immediately improved Jose’s performance as it was the first session in which DCPM (11.63) was higher than DIPM (4.9). CR was administered again as the least effective intervention and resulted in a performance that was level with CCC+RP. During this session, analysis of Jose’s worksheet showed he generalized the area model procedures learned during the previous CCC+RP session that preceded. CCC+RP was identified as the most effective intervention and upon the second administration resulted in a higher DCPM and lower DIPM than CR. Therefore, CCC+RP was verified as the most effective intervention.

Intervention

Jose’s performance on CCC+RP resulted in a stable change in level for DIPM that had no overlapping data points with baseline. DCPM were stable with no increasing or decreasing trend. Analysis of Jose’s DCPM and DIPM during this intervention phase suggested that CCC+RP had helped him acquire accuracy but was seemingly not resulting in improvements in progressing through the second stage of the IH, fluency.

Second BEA

Since Jose’s initially identified intervention and performance thereafter met the criteria outlined in the data analysis section above, a second BEA was administered to determine whether a different intervention was most effective. Jose performed at a level of 20.13 DCPM (0 DIPM) on CR, 30.59 DCPM (0 DIPM) on RP, and 32.26 DCPM (0 DIPM) on CCC+RP. Although CCC+RP resulted in the highest DCPM, the difference from RP was negligible. Due to RP being less intensive than CCC+RP, RP was selected as it was the most efficient intervention. The administration of RP was delayed in favor of baseline worksheets while the paired participant in the multiple baseline design, Don, began his intervention phase. Due to the delay, the BEA was repeated and RP was again identified as the most effective intervention, thus replicating the findings of the second BEA.

Second Intervention

RP was administered for six sessions. Visual analyses indicated an increasing trend in his DCPM and DIPM remained stable at zero. Jose’s final intervention session of 41.1 DCPM (0 DIPM) fell in the mastery range (see Fig. 1).

Don

Baseline

Don averaged 31.06 DCPM (M DIPM = 0.67) during baseline, suggesting he was in the instructional range for two-digit by two-digit multiplication.

BEA

During the CR session Don’s DCPM increased to 43.53 (0 DIPM), whereas, for RP, he performed at 36.39 DCPM (0 DIPM). CR and RP were then alternated for one session each as they were identified as the most and least effective, respectively. These alternated sessions resulted in 46.96 DCPM (2.61 DIPM) for the CR session and 34.15 DCPM (0 DIPM) for the RP session. In order to further verify these results, two additional sessions were administered: CR and RP+. Don performed at 35.81 DCPM (0 DIPM) on CR but only 30 DCPM (0 DIPM) on RP+.

Intervention

CR was implemented in isolation resulting in a more variable but increased level of performance for DCPM as compared to baseline. DIPM was also more variable but remained level with baseline performance. Although DIPM increased, the increase is believed to be largely due to careless errors attempting to complete problems rapidly. Don was already at the upper end of the instructional range for DCPM and had nearly 0 DIPM at the start of data collection, providing little room for improvement. Therefore, Don was more prone to committing errors when completing problems well above the mastery range with some data points reaching as high as 63.87 DCPM (3.87 DIPM).

Evan

Baseline

Evan averaged 15.81 DCPM (M DIPM = 3.48) during baseline, placing him in the frustrational range for two-digit by two-digit multiplication.

BEA

The most effective condition identified by the BEA was CR which produced scores of 25.33 DCPM (0 DIPM; see Fig. 2). The least effective condition was RP (18.86 DCPM; 2.29 DIPM). Alternating these two conditions for one more session verified the results of CR being the most effective intervention.

Intervention

Evan averaged 25.48 DCPM (M DIPM = 3.00) during the CR condition. However, his performance during intervention was highly variable until a sharp upward trend in DCPM and a decreasing trend in DIPM was observed upon the inclusion of his favorite candy to the selection of rewards. His final session resulted in 36.84 DCPM (0 DIPM), which fell in the upper end of the instructional range. Evan’s performance on the generalization probes administered during the intervention phase averaged 27.09 DCPM (M DIPM = 1.6).

Ben

Baseline

Ben’s performance during baseline (M DCPM = 6.67; M DIPM = 15.52) indicated that his two-digit by two-digit multiplication skills were in the frustrational range. Halfway through the baseline phase, his DCPM and DIPM increased in level due to him solving problems more rapidly but continuing to do so inaccurately (M DCPM = 9.24; M DIPM = 19.20).

BEA

CR and RP sessions both resulted in performances similar to baseline with 6.14 DCPM (19.77 DIPM) and 13 DCPM (25 DIPM), respectively. CCC+RP immediately improved Ben’s performance to 18.8 DCPM (0 DIPM). CR, identified as the least effective condition, was administered again and resulted in a DPCM of 17.09 (1.05 DIPM) as Ben generalized the use of the area model similarly to Jose. CCC+RP, the most effective condition, resulted in 24.1 DCPM (0.49 DIPM) on the second administration.

Intervention

Ben’s performance during the CCC+RP condition was relatively stable with greatly increased levels of DCPM and decreased levels of DIPM resulting in neither DCPM nor DICP overlapping with his baseline performance. The data indicated that Ben had moved from the frustrational range to the instructional range for two-digit by two-digit multiplication. The decrease in Ben’s DIPM suggested he had achieved accuracy in the skill, and another intervention may be more appropriate. Therefore, a second BEA was conducted.

Second BEA

The first CR session resulted in 20.89 DCPM (0.89 DIPM), RP in 21 DCPM (0.86 DIPM), and CCC+RP in 16.69 DCPM (2.49 DIPM). CR and CCC+RP were selected as the most and least effective interventions, respectively. However, his performance on the second CR session (21.4 DCPM; 3.26 DIPM) was not as high as the second CCC+RP session (25.34 DCPM; 2.07 DIPM). Due to the lack of replication of findings during the second BEA and external factors preventing the experimenter from working with Ben for several weeks, BEA procedures were conducted again (see Fig. 2). Upon the re-administration of the second BEA, the CR session was identified as the least effective intervention (19.13 DCPM; 0 DIPM) and RP was identified as the most effective intervention (23.70 DCPM; 1 DIPM). Upon alternating these two conditions, Ben performed at 14.61 DCPM (1.55 DIPM) on CR and 23.18 DCPM (0 DIPM) on RP, verifying the identification of RP as the most effective intervention.

Second Intervention Phase

Ben was administered three RP sessions during his second intervention phase in which he averaged 28.77 DCPM (M DIPM = 0.83). His DIPM remained on level with his first intervention phase with CCC+RP, but his DCPM increased in level above the first intervention phase. Ben had his best performance on the final session with 30.68 DCPM (0 DIPM), which fell in the instructional range. Although still in the instructional range, Ben completed 22.9 DCPM (1.83 DIPM) when administered the final generalization probe.

Discussion

To date, studies investigating BEA of academic performance have largely focused on oral reading fluency (Burns and Wagner 2008; Daly et al. 1997, 1999). Of the existing BEA mathematics studies, none employed an array of intervention components that target skill- and performance-based deficits that comprise the IH (Mong and Mong 2012; Reisener et al. 2016). Therefore, consistent with the BEA oral reading fluency research the current study investigated three interventions with components that targeted accuracy and fluency (CCC+RP), fluency (RP), or performance-based (CR) deficits. Overall, BEA procedures were effective in identifying an effective intervention for all four participants as verified by alternating conditions and an extended, isolated implementation of the identified intervention. Each of the two students who engaged in high levels of inaccurate responding (Jose and Ben) had clearly differentiated responding among conditions as CCC+RP represented the first session in which their DCPM was higher than their DIPM. Furthermore, Evan’s data during the CR session were clearly differentiated from the other conditions. Similarly, Don’s data suggested the same differentiation between CR and RP during the alternating sessions. Therefore, the four participants all responded differentially to one of the conditions in the BEA. Consequently, all four participants had at least one condition for which their DCPM increased over baseline. The improvement of DCPM over baseline, however, must be more stringently analyzed and demonstrates the necessity for an error-based dependent variable such as DIPM to be used in conjunction with DCPM.

Solely analyzing student performance according to DCPM paints an incomplete picture for desired improvement. For example, Jose increased in DCPM from baseline during the CR session of his first BEA. However, this improvement represented merely a more rapid completion of problems and not an increase in accuracy as DIPM nearly doubled to 103.33. Alternatively, a failure to consider accuracy can also distort how results are viewed in the opposite direction. Continuing with Jose as an example, his DCPM during the CCC+RP condition in his first BEA was lower than his baseline level and would appear to be an ineffective intervention until DIPM is considered. An examination of DIPM in conjunction with DCPM during those sessions shows that although Jose did not improve in fluency, his accuracy greatly improved. Despite the utility of collecting and graphing both DCPM and DIPM data shown here, the current study and Mong and Mong (2012) are the only two BEA math studies to date to engage in such practices.

Two of the four students (Jose and Ben) were administered a second BEA after continued implementation of the intervention identified as most effective during the first BEA. Both students performed at levels on their second BEAs that suggested they had achieved accuracy in two-digit by two-digit multiplication due to the intervention targeting fluency (RP) being identified as the new, most effective intervention. Changes in the identified intervention demonstrate the sensitivity of BEAs to detect changes in students’ instructional needs and demonstrate the importance of frequently evaluating students’ changing instructional needs.

Results regarding the utility of CCC+RP for improving students’ performance of two-digit by two-digit multiplication with the area model were promising. As CCC+RP was designed with components to improve accuracy, it was identified by BEAs as the most effective intervention for both Jose and Ben, whose baseline data indicated poor accuracy. Both of these participants immediately responded to CCC+RP, resulting in their first sessions in which DCPM was higher than DIPM. Furthermore, Jose and Ben instantly generalized the area model to the next session within the BEA. Before learning the area model in their first CCC+RP session, both Jose and Ben were inaccurate responders with the standard algorithm for two-digit by two-digit multiplication. After learning the area model, they applied it to the problems in the subsequent session even though they received no additional instruction on the method in non-CCC+RP sessions and were not required to employ the procedure. In both cases, Jose and Ben demonstrated higher DCPM than DIPM even though the condition being administered, CR, was identified as the least effective for them. RP was also suggested to be effective with the area model as Jose and Ben continued to develop fluency beyond that achieved by CCC+RP after the implementation of RP.

An additional and important finding of this study is demonstration of the importance of including intervention components for skill- and performance-based deficits. Although teachers were asked to identify students who were inaccurate responders to participate in the study, they selected two students (Don and Evan) who were accurate responders and one student (Don) who was also a fluent responder. By conducting BEAs as opposed to automatically implementing interventions according to teacher hypotheses regarding student deficits, a more appropriate intervention may be selected for a student. As further evidence, RP could be hypothesized to be the most effective for each of these students; however, CR was identified as the most effective intervention for both through BEA. These circumstances display the importance of BEA at its core as an instrument with which educators can objectively and systematically test interventions to determine which will be most effective for remediating a student’s academic responding.

Limitations

Although the current study extends the BEA literature in several ways, it is not without limitations. The most significant limitation of the current study is the lack of a demonstration of incremental validity through an extended analysis between two or more of the interventions. Although such an extended analysis is a component of nearly all previous BEA studies, it was purposefully excluded from the current study in favor of alternating the most and least effective interventions for one more session. An alternating, extended analysis was not conducted due to concerns of generalization when not teaching basic math facts but a method for solving math problems. Since the area model method was being modeled and all students were competent with math facts, they could theoretically solve any problem presented to them by using the model. The result could then be extreme overlap between interventions in an alternating treatment extended analysis. The described concern is evidenced by both Jose and Ben, who immediately generalized the use of the area model to the next BEA session, which resulted in similar performances between interventions. Nevertheless, due to the absence of incremental validity, readers may find the sensitivity of BEAs to changes in student performance to be the single most significant finding of the current study. Second, the range of problem difficulty and the lack of problem sets per intervention might be limitations. Since the problems of each worksheet could range from 13 × 11 to 99 × 99, it was possible for worksheets to vary in difficulty and factor into student performance under conditions. However, each worksheet containing two problems without regrouping and two problems with regrouping constrained the range slightly so that worksheets included two “easier” problems and two “harder” problems. Independent problem sets were not created for each intervention condition, allowing for the potential of carryover and practice effects across conditions. However, the extent of the range of possible problems limited the likelihood of problems being repeated across conditions to a degree that would result in significant threats to internal validity. Finally, conducting two sessions per day could serve as a limitation. Students may have increased their performances on the second session if the first session served as practice for the day. Conversely, students may have become tired of solving problems and performed worse on the second session of the day.

Implications for Practice

As the literature on BEA continues to be advanced and results suggest BEA to be a useful tool for identifying effective interventions for individual students, school personnel should strongly consider its implementation. Results of the current study demonstrate that interventions which target a student’s instructional needs can lead to large and immediate gains. The second BEAs conducted with Jose and Ben also demonstrate the importance of frequently evaluating students’ instructional needs. Once these two students developed accuracy in completing the math problems, it was no longer necessary to provide them with CCC+RP. Continuing to implement the CCC+RP intervention would result in a waste of resources and could potentially hamper their rate of growth. Unfortunately, schools often select an intervention for a student and implement it for up to 10 weeks before deciding whether the intervention was effective. Teachers and other school personnel should attend to both accuracy and fluency when assessing student performance in accordance with the IH. Results of this study provide clear evidence that students’ instructional needs can change frequently, and thus frequent evaluation of their instructional needs should be conducted. The findings of the current study suggest that BEAs could serve as such a vessel for teachers to utilize to evaluate instructional needs in a manner that simultaneously identifies effective interventions.