Introduction

Constructing, comparing, and reflecting on multiple solutions are important instructional elements that are part of high-quality teaching standards in different countries (National Council of Teachers of Mathematics, 2000). Studying the effects of constructing multiple solutions on students’ learning is an important goal for research in mathematics education and has thereby been intensively investigated in the last decade. Yet, such research efforts have focused on the question of how multiple solutions can be taught effectively, and only a few studies have analyzed the effects of constructing multiple solutions on students’ mathematical knowledge-related outcomes (Levav-Waynberg & Leikin, 2012; Star & Rittle-Johnson, 2008). Moreover, there are not many studies on the effects of constructing multiple solutions for real-world problems, despite evidence that the ability to solve real-world problems is important in mathematics education (Niss, Blum, & Galbraith, 2007). Further, the experience of competence has rarely been taken into account in mathematics education research (Schukajlow & Krug, 2014), but it is considered an important educational variable (Deci & Ryan, 2000) that can transmit the effects of constructing multiple solutions (Schukajlow, Krug & Rakoczy, 2015) and prior knowledge (Hänze & Berger, 2007) on knowledge-related outcomes.

In this study, we conducted a field trial to investigate how constructing multiple solutions for real-world problems by applying multiple mathematical procedures affects students’ procedural and conceptual knowledge and their experience of competence and how students’ knowledge and experience of competence are related to each other.

Theoretical Background, Prior Research, and Hypotheses

MultiMa Research Project

This study was conducted as part of the project called Multiple Solutions for Mathematics Teaching Oriented Toward Students’ Self-Regulation (MultiMa; for an overview, see Schukajlow & Krug, 2014). In the first stage of the project, we analyzed how constructing multiple solutions for real-world problems with vague conditions affected students’ achievements, beliefs, emotions, and strategies (Schukajlow & Rakoczy, 2016). In the second stage of the project, we investigated how students’ achievements and perceptions of learning were affected by constructing multiple solutions for real-world problems by applying different mathematical procedures.

The Experience of Competence

According to self-determination theory, every living organism has a system of basic psychological needs that are integrated into a complex system of behavior and motivational control (Deci & Ryan, 2000). These psychological needs are assumed to be important not only for well-being, psychological growth, and integrity (Ryan, 1995) but also for a variety of developmental processes (Deci & Ryan, 2000). Thus, the sufficient fulfillment of psychological needs is a necessary requirement for the optimal functioning of the entire psychological system (Ryan, 1995), and experiences related to the fulfillment of these psychological needs are presumed to play a crucial role in learning (Krapp, 2005). Three essential needs have been postulated: competence, autonomy, and social relatedness. The need for competence refers to “the desire to feel efficacious, to have an effect on one’s environment, and to be able to attain valued outcomes” (Deci, 1998, p. 152). It “encompasses people’s strivings to control outcomes and to experience effectance; in other words, to understand the instrumentalities that lead to desired outcomes and to be able to reliably effect those instrumentalities” (Deci & Ryan, 1991, p. 243). In summary, this basic need is closely related to the inherent satisfaction that results from engaging in activities (e.g., solving tasks during a teaching unit) and extending one’s own capabilities (e.g., learning to solve real-world problems with a new mathematical procedure; Krapp, 2005).

The experience of competence plays a particularly important role in teaching and learning and is prominently featured in social cognitive models of achievement motivation (Wigfield, Battle, Keller, & Eccles, 2002) and in learning theories on self-regulation (Boekaerts & Corno, 2005). It has also been found to enhance achievement (Miserandino, 1996). Furthermore, the experience of competence is closely related to other motivational constructs such as self-efficacy beliefs (Bandura, 2003) because both constructs refer to a person’s cognitions about his or her own ability to perform an action. Whereas the experience of competence refers to how a person feels while working on a problem, self-efficacy expectations refer to the person’s ability to organize and execute a course of action in the future.

Procedural and Conceptual Knowledge

The positive development of students’ procedural and conceptual knowledge is an important component of mathematics achievement (Rittle-Johnson, Schneider, & Star, 2015) and is therefore essential for students’ learning of mathematics (Star, 2007). Procedural and conceptual knowledge represent the ends of a knowledge continuum (Rittle-Johnson, Siegler, & Alibali, 2001). Procedural knowledge can be defined in relation to the skills required to solve problems (Canobi 2009) and thereby involves the ability to execute a series of steps or actions to accomplish a goal (Rittle-Johnson et al., 2015). This type of knowledge “often develops through problem-solving practice, and thus is tied to particular problem types” (Rittle-Johnson et al., 2015, p. 588). On the other end of the continuum, conceptual knowledge is defined as the knowledge of concepts, which are abstract and general principles (Canobi, 2009). Conceptual knowledge involves the comprehension of mathematical concepts, the underlying unifying principles that govern a domain, and their interrelations (Rittle-Johnson et al., 2015).

Effects of Prior Knowledge on the Experience of Competence

Prior procedural and conceptual knowledge are important factors that influence students’ learning. Students with higher achievement in mathematics have been found to report more self-efficacy while working on mathematical tasks than other students (Heinze, Reiss, & Rudolph, 2005). As higher achievement in mathematics is correlated with students’ self-efficacy beliefs (Heinze et al., 2005), and self-efficacy beliefs are closely related to the experience of competence (Bandura, 2003), students’ prior knowledge might be positively related to the experience of competence. Further evidence for this positive relation comes from the finding that students with higher achievement in mathematics report greater interest and more positive emotions while doing math, and both variables are closely connected to the experience of competence (Krapp, 2005). On the basis of this empirical evidence, we expected positive effects of prior procedural and conceptual knowledge on the experience of competence (Hypothesis 1). The relation between prior knowledge and the experience of competence was analyzed in a study in physics (Hänze & Berger, 2007), but this relation has yet to be tested in mathematics.

Effects of the Experience of Competence and Prior Knowledge on Students’ Knowledge

Not only might the experience of competence be influenced by students’ knowledge, but it might also influence their knowledge. Strong feelings of competence are meaningful for students’ intrinsic motivation (Ryan & Deci, 2000), which has a positive influence on academic outcomes (Hattie, 2009). Furthermore, the experience of competence is a valuable part of self-efficacy beliefs (Bandura, 2003), which predict students’ achievements in mathematics (Pietsch, Walker, & Chapman, 2003). In addition, Hänze and Berger (2007) found a direct influence of the experience of competence on students’ achievement in physics, and Schukajlow et al. (2015) confirmed this finding in mathematics. Thus, we hypothesized a positive effect of the experience of competence on students’ knowledge at posttest (Hypothesis 2).

The positive effect of prior knowledge on knowledge at posttest can be hypothesized on the basis of the similar conceptualizations of the pretest and posttest. Both tests assessed students’ conceptual and procedural knowledge. Thus, students who achieved higher scores on the pretest were expected to show higher scores on the posttest also. However, an additional indirect effect of prior knowledge on knowledge at posttest was of interest. As prior knowledge should affect students’ experience of competence during the teaching unit, and their experience of competence should influence their knowledge at posttest, we expected students’ prior knowledge to have an indirect effect on their knowledge at posttest via their experience of competence (Hypothesis 3).

Multiple Solutions and Real-World Problems

Encouraging students to construct multiple solutions for a problem is considered a high-quality element of teaching (e.g., National Council of Teachers of Mathematics, 2000). Constructing multiple solutions and comparing them are important elements of conceptions such as cognitive flexibility theory (Spiro, Coulson, Feltovich, & Anderson, 1988) or cognitive apprenticeship (Collins, Brown, & Newman, 1989). Multiple solutions have been explored in the context of research on problem solving (Becker & Shimada, 1997), and researchers have found that solving problems in more than one way is crucial for constructing comprehensive knowledge (Silver, Ghousseini, Gosen, Charalambous, & Font Strawhun, 2005), fosters the connectedness of knowledge (Levav-Waynberg & Leikin, 2012), helps in the development of a broad range of strategies and representations that foster flexibility (Rittle-Johnson & Star, 2009), and leads to a deeper understanding of the subject at hand (Neubrand, 2006).

One important question in the topic of multiple solutions is how they should be taught in the classroom. Empirical studies have found that students who compared and contrasted different solutions for the same problem made greater gains in procedural knowledge and flexibility than students who reflected on each solution in isolation (Rittle-Johnson & Star, 2007). In addition to this, the practices of reflecting on and discussing multiple solutions were found to be relevant factors that could improve student learning (Rittle-Johnson & Star, 2009).

Most studies on multiple solutions have focused on intra-mathematical tasks (Leikin & Levav-Waynberg, 2007; Levav-Waynberg & Leikin, 2012; Rittle-Johnson & Star, 2007, 2009), but we have focused on the construction of multiple solutions while solving real-world problems. Analyses of problem-solving activities (Blum & Leiss, 2007) have shown that there are different ways to construct multiple solutions while solving real-world problems. We distinguish between three categories of multiple solutions. The first category of multiple solutions results from making different assumptions while solving real-world problems with vague conditions and usually leads to different outcomes. This is the first way to create multiple solutions and was the focus of our previous study. The second category of multiple solutions results from applying different mathematical procedures or strategies to solve a problem and usually leads to the same mathematical outcome. The combination of these two types is also possible and may be considered a third category of multiple solutions. The effects of the second category of multiple solutions are still an open research question and will be addressed in this article.

In the sample problem “Driving School” (Fig. 1), two different driving schools are presented, each with a basic fee and a specific cost for each driving lesson. Students are requested to determine when it is worth going to one school or the other. Two mathematical procedures that are appropriate for solving real-world problems on the topic of linear functions involve (a) the computation of differences and (b) the use of a table.

Fig. 1
figure 1

Real-world “Driving School” task

Full size image

To solve a real-world problem by computing differences, one has to understand the meaning of the important values and to transfer information between reality and mathematics several times. Whereas the basic fee for “Bolls Driving School” is $150 (= $200 − $50) more expensive than the basic fee for “Hilberts Driving School,” each driving lesson at Hilberts Driving School is $5 (= $35 − $30) more expensive than at Bolls Driving School. The question is as follows: How many driving lessons at the more expensive Bolls Driving School does one have to take until the cheaper driving lessons pay off? This occurs after exactly 30 (= $150 / $5) driving lessons. This result has to be interpreted—for example, “Up to 30 driving lessons, Hilberts Driving School is cheaper”—and validated.

Another way to solve this problem is to apply a procedure that involves a table. Students can make assumptions about a possible number of driving lessons (e.g., 10, 20, 40, …), calculate the total cost for both driving schools, compare the costs, identify up to what number of driving lessons one should choose Hilberts Driving School, and write a recommendation for which school is preferable for certain numbers of driving lessons. Although the mathematical procedures can be applied independently, they can also be linked to each other.

Using zero driving lessons as a starting point for the mathematical procedure “table” is equivalent to the difference in the basic fees ($150), which is calculated when using the mathematical procedure “differences.” If a student takes one driving lesson, then the difference between the costs decreases from $150 to $145. In this way, the $5 difference in the cost per driving lesson can be identified, the difference that is also used in the mathematical procedure differences. With each driving lesson, the differences in the costs decrease proportionally. If a student takes five (10, 20, 30) instead of one driving lesson, the difference decreases from $145 to $125 ($100, $50, $0) (Fig. 2).

Fig. 2
figure 2

Linking the mathematical procedures

Full size image

Therefore, in the table procedure, a multistep procedure (× times the $5 difference) is used to identify the number of driving lessons that makes the cost of the two driving schools equal. This is equivalent to the operation used in the mathematical procedure differences in which students calculate the same number of driving lessons in just one step by dividing $150 by $5.

Effects of Constructing Multiple Solutions on Students’ Knowledge and Experience of Competence

The development of multiple solutions allows different solutions to be linked, compared, and reflected on and should therefore foster procedural flexibility (Rittle-Johnson & Star, 2009), which is an important part of deep procedural knowledge (Star, 2007). Constructing multiple solutions should lead to a deeper understanding of the topic at hand (Neubrand, 2006) and thus may improve students’ conceptual knowledge. However, these theoretical considerations have yet to be confirmed empirically. In the study by Star and Rittle-Johnson (2008), students who were instructed to solve the linear equation problems in multiple ways achieved the same procedural knowledge scores on transfer problems as students who were instructed to solve the same problems by applying one mathematical procedure. To the best of our knowledge, no study has explored the effects of constructing multiple solutions on conceptual knowledge. In conclusion, in accordance with the theoretical considerations, we expected that constructing multiple solutions by applying multiple mathematical procedures and emphasizing the links between the mathematical procedures would improve students’ procedural and conceptual knowledge (Hypothesis 4).

An important goal of intervention studies is to identify variables that intervene between treatment conditions and learning outcomes. In research on multiple solutions, students’ experience of competence was expected to act as a transmitting variable for the effects of the treatment condition on motivational and cognitive outcomes (Schukajlow & Krug, 2014; Schukajlow et al., 2015). This idea was confirmed for real-world problems with vague conditions because the number of solutions was found to have positive effects on the experience of competence (Schukajlow & Krug, 2014; Schukajlow et al., 2015). In the present study, we expected that constructing multiple solutions would positively affect students’ experience of competence (Hypothesis 5). Because the experience of competence was expected to predict students’ knowledge at posttest (see Hypothesis 2), we hypothesized indirect effects of constructing multiple solutions on students’ knowledge through the experience of competence (Hypothesis 6).

Path Models and Hypotheses

For an analysis of the links between the treatment conditions (constructing multiple solutions by applying multiple mathematical procedures [MS] vs. one solution by applying one mathematical procedure [OS]), students’ knowledge, and their experience of competence, we applied two path models with the same structure to the data (one for procedural and one for conceptual knowledge in solving real-world problems). This allowed us to test direct effects on students’ experience of competence and their knowledge and also to examine whether the experience of competence would transmit the effects of the experimentally manipulated treatment conditions or cognitive prerequisites on the outcomes. In our path models, we proposed that the total effect of treatment condition or individual prerequisite (MS vs. OS and prior knowledge) on students’ outcomes (knowledge at posttest) would consist of indirect effects (via the experience of competence) and direct effects (direct paths from the treatment condition or prior knowledge to knowledge at posttest).

Our path analytic model was based on theoretical considerations and prior research on the effects of constructing multiple solutions and students’ cognitive prerequisites on students’ knowledge. Because constructing multiple solutions included the application of both mathematical procedures (i.e., table and differences), we analyzed the effects of constructing multiple solutions (MS) against both possible mathematical procedures (table [OS1] and differences [OS2]) (Fig. 3).

Fig. 3
figure 3

Hypothesized path analytic model

Full size image

The following primary hypotheses were tested for procedural and conceptual knowledge:

  1. Hypothesis 1.

    Prior knowledge will positively affect students’ experience of competence (EoC) in mathematics classes.

  2. Hypothesis 2.

    Students’ EoC will positively affect their knowledge at posttest.

  3. Hypothesis 3.

    Prior knowledge will indirectly affect knowledge at posttest such that the positive effects will be transmitted through the EoC in mathematics classes.

  4. Hypothesis 4.

    Constructing multiple solutions will positively affect students’ knowledge at posttest.

  5. Hypothesis 5.

    Constructing multiple solutions will positively affect students’ EoC.

  6. Hypothesis 6.

    Constructing multiple solutions will indirectly affect knowledge at posttest such that the positive effects will be transmitted through the EoC during the teaching unit.

Method

Participants and Procedure

A total of 307 German ninth graders (48.26% female adolescents; mean age = 14.6 years) from four comprehensive schools (German Gesamtschule) that each had three middle-track classes took part in this study. Students were parallelized according to their grades in mathematics. That is, on the basis of students’ grades in mathematics, each of the 12 classes was divided into two parts with the same number of students in each part in such a way that the average level of achievement in the two parts did not differ, and each part was instructed in a separate room. Furthermore, students in the same classes were never assigned to the same treatment condition, and each treatment condition occurred with the same frequency at each school.

In total, there were 24 groups: 8 groups in the MS condition, 8 in the OS1 condition, and 8 in the OS2 condition. Students in the MS condition constructed two solutions, whereas students in the OS conditions constructed one solution only. Before and after the teaching unit, the participants completed tests on procedural and conceptual knowledge in solving real-world problems. After solving the first problem during the second session, students reported their experience of competence during the teaching unit (see Fig. 4).

Fig. 4
figure 4

Overview of the study design

Full size image

Six mathematics teachers between 27 and 60 years of age (three women) instructed students to solve real-world problems in the multiple-solution and one-solution conditions in separate rooms. Each teacher taught the same number of student groups in each treatment condition so the influence of teachers’ personality on students’ learning did not differ between conditions. For the teaching unit, the teachers were given all of the real-world tasks and the solution spaces as well as an instruction manual with lesson plans for each condition.

The teaching unit was implemented during regular classes before students learned to use linear equations and was based on the student-centered method for teaching real-world problems, which has demonstrated a positive influence on students’ learning (Schukajlow et al., 2012). In the MS condition, the first four problems contained the modification “Apply two different mathematical procedures to solve this problem. Write down both procedures.” In the first session, a teacher first demonstrated how real-world problems could be solved by using one specific mathematical procedure (in the OS conditions) or by using multiple mathematical procedures (in the MS condition). Then, students solved two such problems that were based on the student-centered method. After the first problem in the second session in the MS condition, the teacher highlighted and summarized the links between the two mathematical procedures and compared and contrasted the mathematical procedures. In the OS conditions, students solved the same tasks as students in the MS condition and solved an additional task to compensate for the time needed for the comparing and linking of two mathematical procedures in the MS condition. In sum, students in the MS condition solved a total of five problems, whereas students in the OS conditions each solved a total of six problems.

The fidelity of the treatment was ensured by applying the following procedures. The teachers had experience teaching real-world problems and attended a 1-day training session to learn the teaching methods they were expected to apply in the present study. All lessons were observed and videotaped by research group members, and students’ solutions were collected. Analyses of the videos and solutions confirmed that the instruction time did not differ between the conditions and that the teachers provided students with the intended tasks, which required students’ to construct multiple solutions or one solution for real-world problems. In addition, no differences between treatment conditions in the amount of time spent on tasks were reported by the observers. During the first session in the MS condition, students used the time to solve the problems by applying two mathematical procedures, whereas in the OS conditions, students discussed the solution for longer than in the MS condition. Furthermore, as intended, the number of mathematical procedures that were applied by students in the MS condition was significantly higher than in the OS conditions (see Achmetli, Schukajlow & Krug, 2014).

Measures

Tests of Procedural and Conceptual Knowledge.

On the basis of the differentiation of knowledge in the study by Rittle-Johnson and Star (2007), we assessed two types of knowledge with respect to the topic of linear functions in order to test students’ progress. Measures of procedural knowledge almost always involve solving familiar problems, for which the procedures needed to solve them are known (Rittle-Johnson et al., 2015). For the measurement of conceptual knowledge, a large variety of tasks (e.g., explanations of judgments) can be used (Rittle-Johnson & Schneider, 2014). One critical feature of conceptual tasks is that they have to be relatively unfamiliar to participants (Rittle-Johnson et al., 2015).

Students’ procedural and conceptual knowledge in solving real-world problems was estimated by applying a two-dimensional Rasch model (Bond & Fox, 2001). This model allowed us to construct parallel test versions (with no item overlap) for each scale (here, procedural and conceptual knowledge in solving real-world problems) at two measurement points (pretest and posttest). Thus, students’ knowledge could be compared between time points. Students’ procedural knowledge in solving real-world problems was tested with 12 items, and their conceptual knowledge in solving real-world problems was tested with 11 items. Because each student solved similar but not identical items at pretest and posttest, memorization effects were minimized. The ConQuest software (Wu, Adams, & Wilson, 1998) was used to scale students’ data. Weighted likelihood estimator (WLE) parameters (Warm, 1989) were estimated for each student and represented students’ knowledge with continuous scales.

The EAP test reliabilities, which indicate how well the items were spread along the measure in the current sample (Fisher, 1992), were 0.86 for procedural knowledge and 0.68 for conceptual knowledge in solving real-world problems and thus above the typical cutoff value of 0.50 (Rupp, Templin & Henson, 2010). Sample items from the test are presented in Fig. 5. The test as a whole is provided as supplementary material.

Fig. 5
figure 5

Sample items for assessing procedural and conceptual knowledge

Full size image

Because procedural knowledge often develops through problem-solving practice and is thus tied to particular problem types (Rittle-Johnson et al., 2015), all items from this dimension were relatively similar to the tasks students solved during the teaching unit and required students to construct just one solution. Procedural knowledge was assessed with a partial credit model. Students’ responses received a score of 0 for an incorrect solution, 1 if they got the right mathematical result but presented an incorrect interpretation or did not interpret the result, or 2 if both the result and the interpretation were correct. On the conceptual knowledge test, students’ responses were scored 1 for choosing the right answer and providing a correct explanation or 0 for an incorrect solution. In order to assess conceptual knowledge as the comprehension of mathematical concepts with underlying principles and their interrelations (Rittle-Johnson et al., 2015), all items were constructed so that students needed to explain their solutions without executing any mathematical procedure. Therefore, all students who used a calculation to determine the exact solution were given a score of 0.

Experience of Competence Scale.

We used a 5-point Likert scale that ranged from 1 (not at all true) to 5 (completely true). It was comprised of three items that referred to students’ perceptions of the extent to which they were able to implement the given tasks in the actual learning environment. Item 1 (“I noticed that I really understood things”), which came from the study by Hänze and Berger (2007), refers to “understanding the instrumentalities that lead to desired outcomes” (Deci & Ryan, 1991). The second item (“I felt able to master the work”) also came from the study by Hänze and Berger (2007) and refers to attaining valued outcomes (Deci, 1998). The third item (“I felt confident about my knowledge about the topic today”), which we developed in our previous study (Schukajlow & Krug, 2014), refers to a person’s desire to feel efficacious (Deci, 1998).

The internal consistency reliability of the experience of competence scale measured as Cronbach’s alpha was 0.74. This value is comparable to the reliability found in the study by Hänze and Berger (2007), who reported a reliability of 0.82.

Data Analysis

We tested the hypothesized model with regard to procedural and conceptual knowledge in solving real-world problems using the experience of competence as an intervening variable. The treatment factor was dummy coded (MS vs. OS1: 0 = OS1; 1 = MS; MS vs. OS2: 0 = OS2; 1 = MS). To test the hypotheses, we computed two path models with 18 free parameters and 307 subjects. The ratio of subjects to parameters was 17 (307/18), which was above the critical value of 5 that was required for obtaining solid results (Kline, 2005).

Clustering of the Data.

To increase the external validity of the current study, the students were instructed in groups of 10 to 16 students from the same mathematics class rather than individually. To examine the degree of dependence within the groups (n = 24) for prior procedural and conceptual knowledge, we calculated intra-class correlation coefficients (ICCs) using the statistical program Mplus (Muthén & Muthén, 1998–2012) and transformed them into design effects (Muthén & Satorra, 1995) to indicate the loss of statistical power due to the dependence of observations. The resulting design effects of 3.84 for procedural and 2.86 for conceptual knowledge were above the critical value of 2 (Muthén & Satorra, 1995). Furthermore, the between-group variability was significant (p < 0.001) for both types of knowledge. Thus, we calculated fit statistics and assessed the effects using maximum-likelihood estimations with adjusted standard errors (MLR) using the type = complex analysis in Mplus. This statistical method takes into account the dependence of observations for parameter estimates and goodness-of-fit model testing (Muthén & Muthén, 1998–2012).

Missing Values.

The percentage of missing values in the current study differed across the measures from 8.5% for the experience of competence during the teaching unit to 6.5% on the posttest measures. The missing values in the current study were estimated by applying the maximum-likelihood algorithm implemented in Mplus (Muthén & Muthén, 1998–2012). This algorithm uses all of the information from the covariance matrices to estimate the missing values.

Results

Analysis of Fit in Path Models

The correlation matrix of the variables is presented in Table 1. The correlations were in the expected direction.

Table 1 Correlations between all variables
Full size table

The means and standard deviations of the variables are presented separately for each treatment condition in Table 2.

Table 2 Means and standard deviations
Full size table

We applied the combination of cutoff values for the comparative fit index (CFI > 0.95) and the standardized root mean square residual (SRMR < 0.05) to test the goodness of fit of the model. In addition, we also calculated the chi-squared goodness-of-fit statistic. Thus, both path models fit the data well according to all fit indices (see Table 3).

Table 3 Fit values for the path models for procedural knowledge (model PK) and conceptual knowledge (model CK)
Full size table

Tests of Hypotheses

In this section, we present the results of the estimates calculated for the hypothesized path models. Because the treatment conditions represented a binary factor (MS vs. OS1 and MS vs. OS2), StdY values were used to calculate the standardized estimates in Mplus. Thus, β coefficients can be interpreted as the predicted change in (residualized) criterion measures (in standard deviation units) when the treatment changes from 0 (one solution) to 1 (multiple solutions). (Fig. 6)

Fig. 6
figure 6

Graphical illustration of direct effects in the hypothesized path models for procedural and conceptual knowledge. Significant paths (p < 0.05, one-tailed) are illustrated with solid lines and nonsignificant paths with broken lines

Full size image

Effects of Prior Knowledge on the Experience of Competence.

Students with higher prior knowledge felt more competent during the teaching unit (model PK: β = 0.17, SE = 0.05, p < 0.001; model CK: β = 0.21, SE = 0.05, p < 0.001). Thus, students’ prior knowledge was important for their experience of competence during the teaching unit.

Effects of the Experience of Competence on Knowledge at Posttest.

Students’ experience of competence during the teaching unit predicted their procedural knowledge at posttest (β = 0.08, SE = 0.04, p = 0.030) and their conceptual knowledge at posttest (β = 0.17, SE = 0.05, p = 0.001). Students who reported feeling very competent during the teaching unit showed better results for both procedural and conceptual knowledge at posttest.

Effects of Prior Knowledge on Knowledge at Posttest via the Experience of Competence.

As hypothesized, prior conceptual knowledge indirectly positively affected conceptual knowledge at posttest with the experience of competence as an intervening factor (β = 0.04, SE = 0.02, p = 0.018). The indirect effects of prior procedural knowledge on procedural knowledge at posttest with the experience of competence as an intervening variable were marginally significant (β = 0.014, SE = 0.01, p = 0.071). Therefore, we found that students who had a lot of knowledge before the teaching unit and felt more competent during the teaching unit showed improvements in their procedural and conceptual knowledge over and above the well-known effect of prior knowledge. However, the increase in the indirect effects was small compared with the direct effects of prior knowledge.

Effects of Treatment Condition on Knowledge at Posttest.

Contrary to our expectations, we found no total effects of constructing multiple solutions on procedural or conceptual knowledge in solving real-world problems at posttest (model PK: β = 0.01, SE = 0.11, p = 0.451 [MS vs. OS1]; β = − 0.02, SE = 0.10, p = 0.418 [MS vs. OS2]; model CK: β = 0.08, SE = 0.09, p = 0.161 [MS vs. OS1]; β = 0.07, SE = 0.11, p = 0.272 [MS vs. OS2]). Thus, students in the multiple-solution condition did not differ from students in the one-solution conditions in their procedural or conceptual knowledge in solving real-world problems after the teaching unit.

Effects of Treatment Condition on the Experience of Competence.

As expected, constructing multiple solutions positively affected students’ experience of competence (model PK: β = 0.12, SE = 0.06, p = 0.031 [MS vs. OS1]; β = 0.11, SE = 0.04, p = 0.006 [MS vs. OS2]; model CK: β = 0.14, SE = 0.06, p = 0.017 [MS vs. OS1]; β = 0.13, SE = 0.04, p = 0.001 [MS vs. OS2]). Thus, students in the multiple-solution condition reported feeling more competent during the teaching unit.

Effects of Treatment Condition on Knowledge at Posttest via the Experience of Competence.

We found marginally significant positive indirect effects of the treatment on procedural knowledge in solving real-world problems at posttest transmitted via the experience of competence (β = 0.01, SE = 0.01, p = 0.055 [MS vs. OS1]; β = 0.01, SE = 0.01, p = 0.071 [MS vs. OS2]) as well as positive indirect effects of the treatment on conceptual knowledge in solving real-world problems at posttest transmitted via the experience of competence (β = 0.02, SE = 0.01, p = 0.018 [MS vs. OS1]; β = 0.02, SE = 0.01, p = 0.006 [MS vs. OS2]). Constructing multiple solutions for real-world problems provides an opportunity to improve procedural and conceptual knowledge in solving real-world problems indirectly through the experience of competence.

Discussion

The effect of constructing multiple solutions for real-world problems on students’ knowledge is an important issue in mathematics education on the basis of theoretical propositions that constructing multiple solutions should have positive effects on students’ achievements (Levav-Waynberg & Leikin, 2012; Neubrand, 2006; Rittle-Johnson & Star, 2009; Silver et al., 2005), and the ability to solve real-world problems is important for students’ current and future lives (Niss et al., 2007). However, besides case studies, only one randomized experimental study previously investigated the impact of constructing multiple solutions for real-world problems on knowledge (Schukajlow et al., 2015). Moreover, the focus of this previous study was on constructing multiple solutions by making different assumptions about real-world problems with vague conditions, whereas the effects of constructing multiple solutions for real-world problems by applying multiple mathematical procedures on students’ achievement had yet to be investigated, and thus, the current study was conducted to fill this gap.

In the present study, we addressed these research gaps and examined the effects of constructing multiple solutions for real-world problems on students’ knowledge, clarified the role of the experience of competence while solving real-world problems in the development of students’ knowledge, and analyzed the impact of prior knowledge on students’ knowledge at posttest when taking the experience of competence into account. In order to explore the connections between the treatment conditions, students’ experience of competence, and their knowledge, we hypothesized a path model. As expected, the predicted path models provided a good fit to the data for procedural and conceptual knowledge in solving real-world problems. Thus, the models adequately described the influence of the treatment condition and prior knowledge on students’ knowledge at posttest.

Effects of Prior Knowledge on the Experience of Competence

The expected positive influence of prior knowledge on students’ experience of competence was derived from studies in the domains of educational psychology and science education (Hänze & Berger, 2007; Pekrun, Goetz, Frenzel, Barchfeld, & Perry, 2011) and had yet to be analyzed for mathematics education. Our results provided an empirical confirmation of this effect in mathematics education and clearly indicated that students’ prior knowledge has a significant influence on their experience of competence.

Effects of the Experience of Competence and Prior Knowledge on Students’ Knowledge at Posttest

The experience of competence predicted students’ procedural and conceptual knowledge in solving real-world problems as hypothesized on the basis of self-determination theory (Deci and Ryan 2000) and self-efficacy expectancies (Bandura, 2003). This finding confirms empirical results in the same domain (Schukajlow et al., 2015) and in other domains (Hänze & Berger, 2007).

We found evidence for the indirect effects of students’ prior knowledge on students’ knowledge at posttest via their experience of competence as we predicted on the basis of theoretical and empirical considerations from previous studies (Bandura, 2003; Hänze & Berger, 2007; Hattie, 2009; Pietsch et al., 2003; Ryan & Deci, 2000). Higher procedural and conceptual knowledge helps students experience greater competence in the classroom. As a result, students with higher prior knowledge tend to learn more and demonstrate more knowledge at posttest. Given that this indirect effect was nonsignificant when Schukajlow et al. (2015) tested it, the confirmation of this effect in the current study should be acknowledged.

Effects of Constructing Multiple Solutions on Students’ Knowledge and Experience of Competence

Derived from different theoretical assumptions, we hypothesized total effects of constructing multiple solutions on students’ knowledge at posttest. However, the results of our study did not support this prediction. Students in the MS condition and students in the OS conditions were similar in their procedural and conceptual knowledge of how to solve real-world problems using linear functions at posttest after the four-lesson teaching units. This result is in line with the findings from Levav-Waynberg and Leikin’s (2012) study in which no effects of constructing multiple solutions for intra-mathematical problems on knowledge-related measures were found. Similar results were also found in the study by Star and Rittle-Johnson (2008). However, the control group in Star and Rittle-Johnson’s study spent much more time solving different problems with the same standard approach used by the experimental group, whereas in the present study, students in the control group solved only one extra problem to compensate for the time needed to compare and link the two mathematical procedures in the MS condition. A possible reason for our finding is that the problems on the posttest required only one solution, and we argue that there were no differences in knowledge between the groups because one solution is enough when solving problems in real life.

Furthermore, given that a crucial point in the measurement of treatment effects is that the treatment tasks need to provide a good match with the test (Hattie et al., 1996), the way the problems were formulated on the posttest may have negatively impacted the achievement of students in the MS condition in our study because the ability to construct multiple solutions was not captured adequately by the posttest. Other studies have either assessed other aspects of student achievement or have applied different approaches in which problems must be solved in multiple ways, thus offering advantages for constructing multiple solutions for intra-mathematical problems with regard to creativity and procedural flexibility (Levav-Waynberg & Leikin, 2012; Rittle-Johnson et al., 2009). The influence of constructing multiple solutions for real-world problems should be addressed in future studies by presenting problems that require the construction of multiple solutions.

Our findings indicate that the opportunities to link, compare, and contrast different mathematical procedures provided students with feedback regarding their competence and that students felt more competent. However, whether these effects are due to the nature of the problems to be solved or to the design of the corresponding teaching unit remains an open question. Furthermore, students’ experience of competence was found to be a transmitting variable between the treatment condition and knowledge at posttest. These findings extended previous results (Schukajlow et al., 2015). Linking, comparing, and contrasting mathematical procedures should be considered alongside other factors for improving students’ experience of competence during the teaching unit because such procedures can have effects on students’ procedural and conceptual knowledge in solving real-world problems at posttest.

Strengths and Limitations

The effects of the treatment condition and prior knowledge on students’ knowledge at posttest were examined by applying path analyses. The active manipulation of the treatment conditions in our study (MS vs. OS1 and OS2) is a necessary condition for causally interpreting the effect of the treatment on students’ knowledge. However, the validity of the analysis of path models strongly depends on the time points at which the data were collected and on evidence from previous research about the possibilities of the direction of effects such as the impact of the experience of competence on students’ knowledge at posttest. To test the hypothesized path models, we collected data before, during, and after the teaching unit to ensure that the data used in our analyses would be ordered along a timeline. Such ordering of data allows conclusions to be drawn about the direction of influence of measured factors (e.g., from students’ prior knowledge to their experience of competence during the teaching unit) and therefore allowed us to test the hypothesized path models.

The path models were derived from theories about procedural and conceptual knowledge (Rittle-Johnson et al., 2015), self-determination theory (Deci & Ryan, 2000), and approaches for solving real-world problems (Niss et al., 2007) and constructing multiple solutions (Levav-Waynberg & Leikin 2012; Rittle-Johnson & Star, 2007; Schukajlow & Krug, 2014). Theoretical insights and the results of previous empirical studies have supported the hypothesized paths between the applied treatment conditions, the experience of competence, prior knowledge, and knowledge at posttest. However, these path models may be incomplete as students’ prior self-beliefs or other intervening variables such as emotions (Goldin, 2014; Schukajlow & Rakoczy, 2016) or feedback from teachers (Rittle-Johnson & Star, 2009) could affect students’ experience of competence or their knowledge.

We investigated the effects of this intervention across a relatively short time period, and most of the effects were statistically significant but medium or small in size. Although the indirect effects of the intervention via the experience of competence and the direct effects of the experience of competence on procedural and conceptual knowledge were small, they might be practically important for learning because improving students’ experience of competence was found to increase not only cognitive (e.g., knowledge) but also affective (e.g., interest; Schukajlow & Krug, 2014) outcomes. As previously determined, both cognitive and affective outcomes are important for learning. Furthermore, such effects can be expected to be stronger across a longer period of time, and thus, the current approach should be replicated for a longer intervention period. Furthermore, the stability of the effects on knowledge is an important issue, which needs to be addressed with follow-up measures (see e.g., Rittle-Johnson & Star, 2009). Another limitation of the study involves the kinds of mathematical procedures (table and differences) we used because these represent a factor that can influence the effects of constructing multiple solutions on students’ knowledge. The results might be different if other mathematical procedures (e.g., graphical or algebraic) were applied. Moreover, because of the design of our intervention, the number of mathematical procedures implemented in the experimental condition was exactly two; and thus, the term multiple was limited to two solutions in our study. The effects of constructing multiple solutions on students’ knowledge and their experience of competence might be different if a larger number of mathematical procedures could be implemented. Furthermore, future studies should extend phases of linking, comparing, and contrasting different mathematical procedures to strengthen the effects of constructing multiple solutions on students’ knowledge. Additional qualitative analyses of students’ solution processes could help to clarify the role of the experience of competence while solving real-world problems in the development of students’ knowledge as well as to analyze the impact of prior knowledge on students’ knowledge at posttest when taking the experience of competence into account.

Conclusion

Constructing multiple solutions in the mathematics classroom is an important teaching element that has not been investigated very often in experimental studies (Schukajlow et al., 2015). Existing research frameworks usually focus on the implementation of multiple-solution tasks (Leikin & Levav-Waynberg, 2007; Levav-Waynberg & Leikin, 2012) and on the impact of comparisons on students’ learning (Rittle-Johnson & Star, 2007; Star & Rittle-Johnson, 2008). Extending this research, Schukajlow et al. (2015) investigated the effect of constructing multiple solutions by making different assumptions about real-world problems with vague conditions on students’ achievement while highlighting the importance of exploring the effectiveness of constructing multiple solutions for real-world problems on knowledge-related measures. The results of the current study verified that the experience of competence is a factor that transmits the effect of constructing multiple solutions on students’ knowledge at posttest. In addition, the results showed that students’ prior knowledge positively affects their experience of competence, and we found evidence that students’ prior knowledge positively affects their knowledge at posttest via their experience of competence. Therefore, students’ prior knowledge and their experience of competence during a teaching unit are important factors that should be taken into account when teaching students to construct multiple solutions to solve real-world problems.