Introduction

The National Mathematics Advisory Panel (NMAP 2008) reported that U.S. students are only mediocre in math achievement compared to their peers worldwide. This finding is evidenced by the fact that only 23% of students nationwide had achieved math proficiency by Grade 12 (NMAP 2008). Math skill deficits were also found to be the second highest factor in the identification of a learning disability (Kavale and Reese 1992). As NMAP (2008) reports, success in mathematics education is important for all students because it provides them with additional educational options and increases the students’ prospects for future professional careers and earning potential. Therefore, effective interventions are needed to prevent and/or remediate math skill deficiencies.

One intervention designed to address mathematics skill deficits is cover, copy, and compare (CCC). CCC is an effective mathematics intervention designed to improve both accurate and fluent responding across a variety of mathematics calculation skills (Grafman and Cates 2010; Skinner et al. 1989, 1993). CCC is a self-managed intervention that provides a series of learning trials within a short period of time (Skinner et al. 1997b). Overall, CCC has been cited as an effective intervention in addressing mathematics, spelling, and writing skill deficits (McLaughlin and Skinner 1996; Saecker et al. 2009).

The taped-problems (TP) intervention has also been shown to enhance basic fact accuracy and automaticity (McCallum et al. 2004). The TP intervention was adapted from Freeman and McLaughlin’s (1984) taped-words intervention, in which students read word lists along with an audiotape to increase word list reading fluency. Additionally, McCallum et al. (2004) incorporated varying time-delay procedures. Existing research on the taped-problems intervention suggests that it can increase accuracy and fluency when used both at the individual student and group level (Poncy et al. 2007).

Math to mastery (MTM; Doggett et al. 2006) is another intervention that has been shown to increase basic math fluency in applied settings (Hoda 2006; Mong and Mong 2010). MTM is a structured intervention package that includes previewing problems, repeated practice, immediate corrective feedback, summative and formative feedback, and self-monitoring of progress.

Although all three of the above interventions have proven successful in enhancing basic math fluency, selecting the most effective intervention is often difficult because there are many reasons why students may experience academic problems (Daly et al. 1997). Indeed, Lentz and Shapiro (1986) noted that academic problems can be characterized as skill deficiencies, fluency problems, performance problems, or some combination of these factors. Thus, research investigating methods to aid instructional decision making is critical for facilitating the use of empirically-based interventions as well as matching student needs with appropriate instructional interventions (Codding et al. 2007).

One tool, brief experimental analysis (BEA), is a reliable, time-efficient, and cost-effective assessment for students. BEA entails a quick evaluation of interventions using a single-subject design methodology with the intent of determining the most effective intervention procedures for each student (Daly and Martens 1999) and has been shown to predict relatively effective interventions for students with learning difficulties (Daly and Martens 1999; Jones and Wickstrom 2002; VanDerHeyden et al. 2001). Thus, rather than spending time, money, and other resources implementing unsuccessful or moderately successful interventions, this brief assessment procedure allows practitioners to quickly attempt several interventions and assess a student’s response before choosing one intervention for full implementation.

Within the BEA method, a student’s response is first assessed to establish a baseline level of performance with regard to the target behavior. This level of performance is then compared to multiple interventions, intervention components, or combinations of intervention components. The BEA method has been found to identify effective and efficient interventions for reading fluency (Burns and Wagner 2008) in approximately the same amount of time as standardized testing (Jones and Wickstrom 2002).

Despite the success noted in the area of reading, few studies have examined the feasibility of adapting BEA methodology to the area of mathematics (Codding et al. 2009). Additionally, although previous researchers have utilized alternating treatment designs (ATD) to evaluate the efficacy of a BEA-predicted intervention (i.e., Eckert et al. 2000) few, if any, studies have employed an ATD to evaluate the effectiveness of all interventions to validate the BEA prediction. Thus, the purpose of the present study was to evaluate agreement between the results obtained during the BEA and extended intervention analysis phases by comparing all interventions as opposed to only the BEA-predicted intervention. Based on previous research (i.e., Carson and Eckert 2003; Codding et al. 2009; Hendrickson et al. 1996), we hypothesized that the intervention identified to yield the greatest performance (i.e., highest digits correct per minute; DCPM) in the BEA would also be identified as the most effective intervention in the extended intervention analysis. As CCC, TP, and MTM have yet to be compared in a single study, a secondary purpose of this study was to analyze both the efficacy and efficiency of these three math interventions for improving the math fluency of three students identified with math fluency deficits.

Method

Participants and Setting

Participants were three second-grade students from a suburban school in the Southeastern United States with approximately 520 students (grades pre-kindergarten-4), 42% of whom qualified for free or reduced-price lunch and 45% of whom were from ethnic minority backgrounds. Cooper was a 7-year-old Caucasian boy, Rebecca was an 8-year-old Caucasian girl, and Duke was an 8-year-old African American boy. The students who participated in the study provided student assent, and parental consent was obtained consistent with procedures outlined by the university Institutional Review Board.

Additionally, none of the students was receiving special education services or other mathematics interventions according to school records. The students for this study were identified based on teacher recommendation and by their performance in math as measured by three curriculum-based measurement (CBM) probes previously administered during local norming assessments. Three different second-grade level multiple-skill probes were used. The local norming probes were generated using the Curriculum-Based Assessment Math Computation Probe Generator available on the Web site http://www.interventioncentral.org (Intervention Central n.d.) and were timed for 2 min. State benchmarks were used to determine which skills were taught at each grade level. Based on district-level identification procedures using local norms, the participants scored in the lowest quartile. The primary author served as the interventionist and implemented all intervention and progress monitoring procedures. All sessions were conducted in an empty classroom.

To assess the possibility that the participating students’ math fluency deficits were the result of a motivational deficit as opposed to a skill deficit, contingent reinforcement (CR) was implemented prior to the study beginning. Each student was asked to provide his or her classroom teacher with a list of five items that he or she was willing to work for if progress was made (e.g., candy, stickers, bookmarks). Each student was then given three multiple-skill grade-level probes with the median score serving as the CR baseline. At the beginning of each CR session, each student’s teacher showed the student a grab bag that contained the five items that the student stated he or she would work for. Each teacher then explained to the students that they could choose an item from the grab bag contingent on obtaining more digits correct than baseline. After the student completed the probe, the teacher calculated his or her progress. If the student increased his or her DCPM by 10% over baseline, he or she was allowed to choose an item from the grab bag. Previous research has noted that many students exhibiting performance problems may benefit from CR (Morgan and Sideridis 2006). However, none of the three students in this study increased his or her DCPM by the required 10% over baseline during the 2 week CR intervention. Thus, further interventions targeting fluency deficits were needed.

Materials

Baseline, BEA, and Extended Intervention Analysis Worksheets

Calculation problems were generated using a web-based computer program, Math Worksheet Generator, available on the Web site http://www.interventioncentral.org (Intervention Central n.d.). The program allows the user to design worksheets requiring the use of specific skills. State benchmarks were used to determine which skills were representative of each grade level. The program was then used to create seven sets of worksheets for each student, across conditions, specific to a particular grade level and state benchmark. The computer program randomized (a) the order of problems within a worksheet and (b) the order of the factors within each problem. All worksheets included a number to identify the worksheet within the grade level and blank lines for name, date, and examiner. None of the worksheets overlapped during intervention.

Baseline, TP, and MTM extended intervention analysis instructional worksheets contained 24 problems of the targeted single skill with six rows of four problems. For each CCC extended intervention analysis worksheet, four calculation problems with correct answers were listed on the left side of the worksheet, whereas the same calculation problems without answers were listed on the right side of the worksheet. For each session, six CCC worksheets (24 total problems of the targeted single skill), consisting of different problems, were used.

The specific instructional skills identified for remediation were as follows: Cooper; addition with two, two-digit numbers with regrouping (i.e., 29 + 11), Rebecca; addition with one, two-digit number plus one, one-digit number with regrouping (i.e., 52 + 9); Duke; addition with one, two-digit number plus one, one-digit number without regrouping (i.e., 11 + 4). The specific skills chosen for each student were based on error analysis (frequency of errors on a particular skill set) of CBA probes and confirmed by the students’ respective classroom teachers (Mong and Mong 2010).

Generalization Worksheets

Generalization worksheets were identical in creation to baseline and extended intervention analysis worksheets with the exception that they were comprised of 24 mixed-skill grade-level problems based on state department of education benchmarks rather than the targeted single skill. Problems included the following: addition with two, one-digit numbers with and without regrouping (i.e., 1 + 7, 6 + 9), addition with one, two-digit number and one, one-digit number without regrouping (i.e., 22 + 5), addition with two-digit numbers without regrouping (i.e., 10 + 15), subtraction with two, one-digit numbers (i.e., 5 − 1), subtraction with one, two-digit number and one, one-digit number without regrouping (i.e., 19 − 6), and subtraction with two, two-digit numbers without regrouping (i.e., 24 − 12).

Dependent Measures and Scoring Procedures

The number of digits correct per min (DCPM) served as the primary dependent variable for this study. A digit was scored as correct when the appropriate number was written in the proper column (Shinn 1989). Based on guidance from Shapiro (2004), DCPM were calculated by dividing the number of digits correct by the total number of seconds and multiplying by 60. Errors per min (EPM) served as a secondary dependent variable. Responses were scored as errors if incorrect digits were written below the sum or difference line or if digits were written in the wrong place or omitted.

When evaluating math interventions, an important factor for consideration is the efficiency involved in each intervention. Intervention efficiency was calculated by determining the difference between the mean follow-up DCPM for each intervention and mean baseline DCPM. This increase in rate was then divided by the total time spent in each intervention (5 min in CCC, 3 min in TP, 10 min in MTM multiplied by six sessions).

Experimental Design and Procedures

An alternating treatments design (ATD) was used during the BEA. Experimental conditions (CCC, TP, and MTM) were each presented on a single occasion. Following implementation of each condition, the most effective intervention was identified as the one producing the highest DCPM based on visual inspection. In addition, a condition was deemed effective if a 20% or greater increase in performance over the baseline median was obtained (Jones and Wickstrom 2002; Noell et al. 2001).

An ATD was also employed during the extended analysis to compare baseline performance with performance under the selected and unselected interventions across target and generalization worksheets. Only one intervention was implemented per day. Intervention conditions were counterbalanced to control for potential carryover and order effects. Differentiation between conditions was considered evidence of experimental control (Barlow and Hersen 1984).

The percentage of non-overlapping data points (PND) between conditions was also calculated to supplement visual inspection (Olive and Smith 2005). PND was calculated by dividing the number of non-overlapping data points with baseline by the total number of intervention data points in each condition. As this study was designed to improve student academic skills, the highest baseline data point was used to establish the overlap of baseline data points with intervention data points. Scruggs and Mastropieri (1998) established benchmarks for PND interpretation. Specifically, PND scores below 50% suggest an ineffective intervention, PND scores between 50 and 70% suggest a questionably effective intervention, PND scores between 70 and 90% suggest an effective intervention, and PND scores above 90% suggest a very effective intervention.

Curriculum-Based Assessment (CBA)

The students’ instructional levels on three equivalent mixed-skill grade-level probes were established using standardized CBA procedures (for specific guidelines, please see Shapiro 2004). According to Burns et al. (2006), a student’s independent instructional level was the point in the curriculum where he or she could complete math problems with 14–31 digits correct if enrolled in Grades 1–3. Mastery performance was achieved if a student obtained 32 or more digits correct. All students’ instructional levels were identified as second grade; Cooper obtained 22 DCPM with 5 EPM, Rebecca obtained 17 DCPM with 4 EPM, and Duke obtained 14 DCPM with 3 EPM.

Baseline

During each session, students attempted to complete one CBM probe consisting of 24 problems in the targeted single skill and one CBM probe consisting of 24 multiple-skill grade-level problems (i.e., addition and subtraction with one- and two-digit numbers with and without regrouping) (Shinn 1989). Students received no intervention during the baseline phase. Each baseline probe was administered for 2 min.

Brief Experimental Analysis (BEA)

A BEA was conducted following a stable baseline to examine the effects of the selected intervention conditions on math fluency for each student. The BEA consisted of administering each intervention one time (Wilber and Cushman 2005). The order of interventions in the BEA was randomly selected and all three were administered in 1 day. A 1-h break was provided following each intervention session to avoid practice effects. On average, CCC sessions lasted approximately 5 min, TP lasted 2 min, and MTM lasted 10 min.

Extended Intervention Analysis

Each intervention condition was presented in a counterbalanced order such that no one intervention was followed by that same intervention. Each intervention was administered once per day before lunch. All interventions were implemented using standard procedures outlined in previous research (i.e., Carroll et al. 2006; Doggett et al. 2006; Poncy et al. 2007; Skinner et al. 1997a).

Cover, Copy, and Compare Intervention

Cover, copy, and compare Intervention (CCC) consisted of five steps (Skinner et al. 1997b): (a) look at the problem with the answer, (b) cover the problem with an index card, (c) write the solution to the problem without the answer, (d) uncover the problem and solution, and (e) compare your answer. If the students’ answer did not match the pre-recorded answer, students were told to repeat the steps. Prior to implementation, the examiner explained all five steps and demonstrated the steps before the first session. CCC sessions ended when each student correctly completed all 24 problems. On average, these sessions lasted approximately 5 min. A 2-min CBM probe in the target skill and a 2-min CBM mixed-skill grade-level generalization probe were administered using standard instructions (Shinn 1989) after the 5-min intervention session.

Taped-Problems Intervention

The TP intervention consisted of the following steps (Carroll et al. 2006; Poncy et al. 2007). All students listened to an audio recording of basic math fact problems. Initially, the audio recording provided the correct answer immediately (and the calculation procedure for problems that involved regrouping) after the problem was given. After the initial probe, a time delay between the problem and the answer given on the recording was introduced. This time delay was gradually decreased to promote rapid responding (i.e., 3 s to 2 s to 1 s delays). During the delay, the students tried to “beat” the tape by correctly answering each problem before the tape provided the answer. In the BEA, the students were only given the 3-s delay condition. TP sessions ended after each student completed all 24 problems. On average, the TP sessions lasted 3 min for each participant. A 2-min CBM probe in the target skill and a 2-min CBM mixed-skill grade-level generalization probe were administered using standard instructions (Shinn 1989) after the intervention.

Math to Mastery Intervention

The following steps of intervention were based on the Math to Mastery intervention manual developed by Doggett et al. (2006). The examiner completed each math problem while describing every step of the computation procedure as the student followed along. Subsequently, the student practiced completing each math problem on the math probe in a series of 1-min trials until a mastery criterion of 31 digits correct was obtained or until 10 trials were attempted. While the student was completing each problem, the interventionist followed along, marking digits in error and giving immediate corrective feedback. Immediately after each 1-min math trial, the examiner calculated and informed the student of his or her DCPM for the trial while also offering praise for effort and performance on each trial and recording DCPM and EPM. The student then completed a self-monitoring chart at the end of each 1-min math trial to visually display his or her ongoing performance. On average, the MTM sessions lasted 10 min for each participant. A 2-min CBM probe in the target skill and a 2-min CBM mixed-skill grade-level generalization probe were administered using standard instructions (Shinn 1989) following the intervention.

Generalization

To assess the predictive validity of the BEA in selecting interventions that may generalize beyond targeted single skills, generalization of students’ math fluency was assessed using multiple-skill grade-level probes. Collection of generalization data began on the second day of targeted single-skill intervention. All generalization sessions were conducted after lunch such that each student received a targeted single-skill session before lunch and a multiple-skill session after lunch.

Maintenance Probes

To examine maintenance of skills, follow-up data were collected 5 days and 15 days after the last intervention day. All follow-up data sessions took place the same day with interventions occurring in the same order as the BEA. Two novel single-skill instructional probes were used to assess maintenance of the targeted skill.

Treatment Integrity

Treatment integrity checklists for baseline, BEA, and each intervention condition were developed to standardize and maintain administration fidelity. Observers marked all items on the treatment integrity checklists that were correctly completed. A second observer was used to assess treatment integrity for a minimum of 45% of the sessions evenly distributed across all phases of the study. The second author, who was a doctoral-level practitioner with several years of experience with math interventions, served as the second observer for all treatment integrity checklists.

Treatment integrity was calculated by dividing the number of items on the checklist completed appropriately by the number of items on the checklist and multiplying by 100%. Treatment integrity was assessed across 57% of all sessions. Treatment integrity means were as follows: (a) baseline—100%; (b) BEA—99% (range, 99–100%); and (c) extended intervention analysis conditions; CCC—98% (range, 98–100%), TP—96% (range, 94–97%), and MTM—96% (range, 93–97%). Overall mean treatment integrity for all conditions in this study was 98% (range, 94–100%).

Reliability

Interscorer Agreement (ISA)

The primary author designed a list of scoring instructions for the mathematics probes used in the investigation. ISA was calculated by dividing the number of agreements per digit by the number of agreements plus disagreements per digit and multiplying by 100%. Thirty-six percent of the total probes were independently scored by the primary and secondary authors across all phases of the study. Both authors were doctoral-level school psychologists with several years of experience with math interventions. The mean ISA was: (a) baseline—98% (range, 97–100%); (b) BEA—99% (range, 97–100%); and (c) intervention conditions; CCC—97% (range, 95–100%), TP—98% (range, 96–99%), and MTM—99% (range, 98–100%). Overall ISA for this study was 98% (range, 97–100%).

Interobserver Agreement for Integrity

Interobserver agreement (IOA) for treatment integrity was calculated by dividing the agreed-upon number of steps completed for each session by the number of available steps to complete for each session and multiplying this ratio by 100%. Mean IOA for each phase was as follows: (a) baseline—100%; (b) BEA—98% (range, 96–99%); and (c) intervention conditions; CCC—94% (range, 92–100%), TP—95% (range, 93–98%), and MTM—93% (range, 90–96%). Overall IOA for treatment integrity was 96% (range, 92–100%).

Social Validity

To measure social validity of the interventions, students were administered a seven-item questionnaire adapted from the Children’s Intervention Rating Profile (CIRP) (Turco and Elliott 1986). The CIRP was also modified to assess students’ acceptability of math interventions in previous studies (e.g., Arra and Bahr 2005; Mong and Mong 2010). The questionnaire consisted of seven items rated on a Likert scale of 1 (strongly disagree with the statement) to 6 (strongly agree), with the highest total acceptability being 42.

Results

BEA

Figure 1 displays the BEA results for all three students. The MTM intervention was the most effective intervention for two students, whereas CCC was the most effective for the third student based on single BEA worksheet performance. Cooper achieved his highest DCPM and lowest EPM under MTM. Under MTM, Cooper’s performance exceeded his median baseline by 22% and was slightly more effective than CCC (17% greater than median baseline). Rebecca achieved her highest DCPM and lowest EPM under CCC. Under CCC, Rebecca’s performance exceeded her median baseline by 59% and was slightly more effective than MTM (47% greater than median baseline). Similar to Cooper, Duke achieved his highest DCPM and lowest EPM under MTM. Under MTM, Duke’s performance exceeded his median baseline by 53% and was more effective than CCC (40% greater than median baseline). The TP intervention was observed to be the least effective for all three students in the BEA phase, although it should be noted that Rebecca and Duke’s performance increased by 20% or more over their respective median baseline while under TP.

Fig. 1
figure 1

DCPM and EPM for Cooper, Rebecca, and Duke on baseline and BEA phases

Full size image

Extended Intervention Analysis

Figure 2 displays the results of the target skill extended intervention analysis for all three students. Divergence from baseline and increases in both trend and level were observed for all three intervention conditions across students. All students were able to achieve multiple mastery performance points across all three interventions. All three interventions were also effective at decreasing errors.

Fig. 2
figure 2

DCPM and EPM for Cooper, Rebecca, and Duke on extended intervention analysis targeted single-skill probes

Full size image

Cooper achieved a mean of 24.3 (range, 23–26) DCPM in baseline, a mean of 32.5 (range, 27–38) DCPM in CCC, and a mean of 28.3 (range, 25–31) DCPM in TP. He achieved mastery scores on 4 of the 7 (57%) probes under both CCC and MTM and achieved a mastery score on 1 of the 7 (14%) TP probes. Cooper achieved his highest median DCPM data point (33.5) and highest mean DCPM (34; range, 29–40) under the MTM condition. Overall, Cooper’s mean performance under both MTM and CCC surpassed the mastery threshold of 32 DCPM. The PND for MTM and CCC was 100%, indicating a very effective intervention effect, while the PND for TP was 67%, indicating a questionable intervention effect. Cooper maintained mastery performance during all maintenance sessions across all three interventions.

Rebecca achieved a baseline mean of 22.0 (range, 20–24) DCPM, an MTM mean of 28.2 (range, 24–34) DCPM, and a TP mean of 25.3 (range, 22–29). She achieved mastery scores on 2 of the 7 (29%) CCC probes, 1 of the 7 MTM probes (14%), and 0% of the TP probes. Rebecca achieved her highest median DCPM (30.5) and highest mean DCPM (30.5; range, 24–37) under the CCC condition. Overall, Rebecca did not achieve mastery under any of the interventions based on her mean DCPM. Rebecca maintained mastery performance under both CCC maintenance sessions and the first MTM maintenance session. The PND for both CCC and MTM was 83%, indicating effective intervention effects, while the PND for TP was 50%, indicating an ineffective intervention effect.

Duke achieved a baseline mean of 17.2 (range, 15–19) DCPM, a CCC mean of 21.5 (range, 17–27), and a TP mean of 23.0 (range, 18–29) DCPM. He achieved mastery performance on 1 of the 7 (14%) MTM probes and failed to achieve mastery performance on any probe under CCC or TP. Similar to Cooper, Duke achieved his highest median DCPM (25.5) and highest mean DCPM (25.2; range, 17–32) under the MTM condition. Duke was able to maintain mastery performance only under MTM. Overall, Duke did not achieve mastery under any of the interventions based on mean DCPM performance. The PND for MTM was 83%, indicating an effective intervention effect, and 67% for both CCC and TP, indicating questionable intervention effects.

With regard to errors, Cooper displayed a gradual decrease in both level and trend across interventions as compared to baseline. Although all three interventions reduced his EPM, the CCC and MTM interventions were more effective than TP. Cooper committed a mean of 2.8 EPM (range, 1–4) during baseline, a mean of 1.0 EPM (range, 0–3) under CCC, a mean of 2.5 EPM (range, 1–4) under TP, and a mean of .83 EPM (range, 0–2) under the MTM intervention. During the follow-up phase, Cooper maintained low error rates, only committing one error in each TP follow-up.

Rebecca displayed a gradual decrease in both level and trend across interventions as compared to baseline. She committed a mean of 2.2 EPM (range, 1–3) during baseline, a mean of 1.2 EPM (range, 0–3) under CCC, a mean of 2.8 EPM (range, 1–4) under TP, and a mean of 2.0 EPM (range, 0–4) under the MTM intervention. During the follow-up phase, Rebecca maintained low error rates by committing one error in the second CCC follow-up, one error in the first and two errors in the second TP follow-up, and one error in each MTM follow-up.

Duke displayed a gradual decrease in both level and trend across interventions as compared to baseline. He committed a mean of 3.2 EPM (range, 1–3) during baseline, a mean of 2.7 EPM (range, 2–4) under CCC, a mean of 2.2 EPM (range, 0–4) under TP, and a mean of 1.3 EPM (range, 0–3) under the MTM intervention. During the follow-up phase, Duke maintained low error rates by committing no errors in the TP and MTM follow-ups, and two errors in the first CCC, and one error in the second CCC follow-up.

Intervention Efficiency

Cooper increased .42 digits per min (25.2 digits increase per hour of instruction) under CCC, .43 digits increase per min (25.8 digits per hour) under TP, and .26 digits increase per min (15.7 digits per hour) under MTM. Rebecca increased .42 DCPM under CCC (25 digits per hour), .22 DCPM (13.3 digits per hour) under TP, and .16 DCPM increase (9.6 digits per hour) under MTM. During CCC, Duke increased .26 DCPM (15.6 digits per hour), .54 DCPM (32.7 digits per hour) under TP, and .24 DCPM (14.3 digits per hour) under MTM. Using this method of estimating efficiency, Cooper and Duke achieved their greatest increases in DCPM under the TP intervention, while Rebecca achieved her greatest increase under CCC.

Generalization

Figure 3 displays the results of the generalization extended intervention analysis for all three students. Visual analysis revealed divergence from baseline for all three intervention conditions across students. Cooper achieved his highest DCPM (31), highest mean DCPM (24.8; range, 20–31), and fewest mean EPM (1.7; range, 0–3) under the MTM condition. The PND for all three intervention conditions was 67%, indicating questionable intervention effects. Rebecca achieved her highest DCPM (32) and lowest mean EPM (1.5; range, 0–3) under both the MTM and CCC conditions. She achieved her highest mean DCPM (26; range, 21–32) under the MTM condition. The PND for MTM was 100%, indicating a very effective intervention; 83% for TP, indicating an effective intervention; and 67% for CCC, indicating a questionable intervention effect. Duke achieved his highest DCPM (25), highest mean DCPM (19.8; range, 15–25), and fewest mean EPM (1.3; range, 0–3) under the MTM condition. The PND for both MTM and CCC was 67%, indicating questionable intervention effects, and 50% for TP, indicating an ineffective intervention.

Fig. 3
figure 3

DCPM and EPM for Cooper, Rebecca, and Duke on multiple-skill generalization probes

Full size image

With regard to errors, all three participants displayed a gradual decrease in both level and trend across interventions as compared to baseline. Unsurprisingly, all of the participant’s EPM were higher in the mixed-skill generalization grade-level probes than the targeted skill instructional probes. Cooper obtained a mean of 3.5 EPM (range, 2–5) during baseline, a mean of 2.8 EPM (range, 1–4) under CCC, a mean of 2.8 EPM (range, 1–5) under TP, and a mean of 1.7 EPM (range, 0–3) under the MTM intervention. Rebecca obtained a mean of 2.7 EPM (range, 1–5) during baseline, a mean of 1.5 EPM (range, 0–3) under CCC, a mean of 2.3 EPM (range, 1–3) under TP, and a mean of 1.5 EPM (range, 0–2) under the MTM intervention. Duke obtained a mean of 3.5 EPM (range, 2–5) during baseline, a mean of 2.5 EPM (range, 1–4) under CCC, a mean of 3.3 EPM (range, 2–5) under TP, and a mean of 1.3 EPM (range, 0–3) under the MTM intervention.

Social Validity

Total acceptability scores on the modified CIRP for the CCC intervention were 32 (Cooper), 29 (Rebecca), and 30 (Duke) of 42. Cooper and Duke strongly agreed that they liked the CCC intervention, while Rebecca remained neutral toward CCC. All students felt that the CCC intervention would help them in school, and Cooper did not think there were better ways to help improve his math fluency. Total acceptability scores on the modified CIRP for the TP intervention were 36 (Cooper), 38 (Rebecca), and 36 (Duke) of 42. All students strongly agreed that they liked the TP intervention, felt that the TP intervention would help them in school, and Rebecca and Duke did not think there were better ways to help improve their math fluency. Total acceptability scores on the modified CIRP for the MTM intervention were 32 (Cooper), 30 (Rebecca), and 31 (Duke) of 42. All three students agreed that they liked the MTM intervention and felt that the MTM intervention would help them in school.

Discussion

The purpose of this study was to evaluate agreement between the results obtained during the BEA and extended intervention analysis phases by comparing all three interventions as opposed to only the BEA-predicted intervention. Similar to the findings of previous research (i.e., Carson and Eckert 2003; Codding et al. 2009; Hendrickson et al. 1996), the results of the current study suggest that BEA can effectively predict the most effective intervention for increasing student math fluency. For all three students, the BEA-predicted most effective intervention was congruent with the results of the extended intervention analysis of the targeted single skill. The BEA correctly predicted MTM as the most effective intervention for Cooper and Duke and CCC as the most effective intervention for Rebecca. Furthermore, the results of the extended intervention analysis of grade-level multiple-skill generalization probes were consistent with the predictions of the BEA for two of the three students (Cooper and Duke) as the MTM intervention was the most effective across both phases. However, the BEA incorrectly identified CCC as the likely most effective intervention for generalization. Rebecca’s extended intervention analysis of generalization revealed that MTM was the most effective intervention. Overall, these findings support the utility of BEA procedures for identifying the appropriate intervention for each student.

A secondary purpose of this study was to analyze the efficacy and efficiency of the CCC, MTM, and TP interventions to improve the math fluency of three students identified at the instructional stage of learning. The results from the extended intervention analysis suggest that all three interventions were effective in increasing the students’ math fluency. However, MTM was the only intervention in which all three students achieved mastery performance on individual probes. In comparison, both Cooper and Rebecca achieved mastery performance on multiple probes under CCC, while only Cooper achieved mastery performance on individual probes under all three interventions. This finding is supported by the PND calculations as the MTM intervention was the only intervention deemed to be “very effective” or “effective” across all students. CCC was “very effective” with Cooper, “effective” with Rebecca, and had a questionable intervention effect with Duke. TP had a “questionable” intervention effect on Cooper and Duke and was “ineffective” with Rebecca. Furthermore, an outcome measure that might be particularly meaningful in mathematics is retention of previously mastered skills (VanDerHeyden and Burns 2009). MTM was the only intervention under which all three participants were able to maintain mastery performance on at least one maintenance probe. Perhaps the relative success of MTM is due in part to the notion that it was the most closely linked intervention to the specific skill being assessed. Specifically, MTM involved 1-min trials with a mastery criterion associated with reinforcement. Among the interventions, MTM was most closely aligned to the skill assessed by the dependent measure (mathematics fluency as measured by DCPM) and mirrored the assessment context of a brief timed exam. This also seems to best explain the generalization data, whereas improvements may be a result of completion rate, rather than generalization across problem type.

With regard to the extended intervention analysis of generalization, all three students showed improvement across intervention conditions. Both Cooper and Rebecca were able to achieve mastery scores on a single generalization probe under MTM, while Rebecca also achieved mastery on a single CCC generalization probe. Despite his gains, Duke was unable to achieve mastery under any of the intervention conditions on the generalization probes. None of the participants achieved mastery criteria based on mean performance across seven generalization probes.

Despite the increases noted in DCPM and decreases in EPM for all participants across intervention conditions, it should be noted that only Cooper achieved mastery based on mean performance across seven intervention sessions. This finding combined with the relatively stable baselines may indicate limited general skill acquisition. Future researchers should consider lengthening the extended analysis phase to evaluate both divergence among intervention conditions and the probability of skill acquisition.

The participant ratings of social validity of the interventions were likely affected by the time required and the perceived ease of implementation regarding each intervention. MTM took approximately twice as long on average to implement as compared to CCC, and approximately three times as long as the average time for TP. This finding should be noted as teacher time required for both CCC and TP is significantly less than the time required to implement MTM. Furthermore, both CCC and TP may be implemented as group interventions, whereas MTM was designed for individual intervention. This may be one reason why all three participants rated TP as the most likeable intervention. Interestingly, all three participants rated the MTM intervention as more or equally likeable than CCC despite the additional time and interventionist involvement required with MTM. These findings may indicate that an intervention that demands additional costs may be recommended for students with math deficits to gain sufficient levels of benefits in typical school settings where resources are limited.

Perhaps the CCC and TP interventions are better suited to students at higher instructional levels given that these students may be more knowledgeable about basic math operations and may require less intense and more efficient interventions to achieve mastery. Conversely, the MTM intervention may be more suitable to students at lower instructional levels or students at the frustrational level. These students likely require repeated practice that is crucial for building automaticity with calculation deficits (Hasselbring et al. 1988). During MTM, students repeatedly complete the same worksheet until they reach mastery level before intervention data are collected, whereas similar repeated practice components are not present in either CCC or TP.

Although the present study contributes to the understanding of the effects of CCC, TP, and MTM on math fluency as well as the utility of BEA for mathematics, several limitations should be noted. First, threats to internal validity (e.g., maturation and ongoing class instruction) and threats to external validity (e.g., three students in general education) should be stated. Replication of this study with diverse populations (e.g., age, grade, gender, and special education classification) may address these issues.

A second limitation may be related to the methodology used in conducting the BEA and extended intervention conditions. All BEA and intervention sessions were conducted outside the classroom using trained researchers. Although this may have increased the study’s internal validity, it somewhat limits the generalizability of the findings to use of the BEA and interventions by personnel without such training. Due to concerns associated with the treatment integrity of interventions conducted by personnel who may not be familiar with the procedures (e.g., Wickstrom et al. 1998), future studies should evaluate the level of consultation and assistance needed to train practitioners or other school personnel to conduct BEAs and implement the intervention procedures used in the current study. However, it should be noted that CCC, TP, and MTM have been shown to be easily mastered by teachers with no previous training in CBM procedures.

Third, the results are limited in their generalizability to mathematic skills other than the basic addition and subtraction facts used in the current study. Future research should examine and compare the effects of all three interventions on other areas of math such as multiplication and division.

A fourth limitation is the research design implemented in the extended intervention analysis. The ATD is effective in evaluating outcomes in a brief period of time as all conditions can be evaluated within the same phase. However, multiple-treatment interference is always a concern with this design as the students were exposed to all intervention conditions (Hayes et al. 1999). However, no-training control probes were conducted in an attempt to evaluate potential multiple-treatment interference effects.

Although the interventions used in the present study are representative of empirically-supported school-based interventions, other interventions (i.e., explicit timing, Carson and Eckert 2003; Van Houten and Thompson 1976; interspersal, Skinner et al. 1999, and modified cover, copy, and compare, Stading et al. 1996) were not included. Future researchers should also consider evaluating the effects of these interventions against each other.

As CCC, TP, and MTM are multi-component instructional procedures, future researchers should also conduct component analysis studies to determine which component or combination of components caused the increases in accuracy and fluency. Additionally, researchers should consider how manipulating components can enhance fluency rates.

In summary, the CCC, TP, and MTM interventions were successful in increasing computation fluency for all three students compared to baseline. However, the results indicate that MTM was the most effective intervention for two students while CCC was the most effective intervention for one student. It should be noted that the TP intervention was the most efficient for two of the three students. With regard to the BEA, the results of the current study suggest that the BEA effectively predicted the most effective math intervention for each individual student in the extended analysis of the targeted single skills. Thus, it appears that the BEA is an efficient procedure that allows practitioners to quickly employ several math interventions to determine the most effective intervention to fully implement for each student.