Keywords

The development of adaptive expertise with arithmetic has long been stated as foundational for developing proficient future skills in all areas of mathematics and is a goal of curricula all over the world (National Council of Teachers of Mathematics, 2000). By placing too much focus on training solution algorithms, traditional arithmetic instruction has often been criticized as encouraging the development of routine rather than adaptive expertise, where students become highly efficient in applying different algorithms for very specific types of problems but fail to transfer these skills into new contexts and problem types (Blöte, Klein, & Beishuizen, 2000; Hatano & Oura, 2003).

Adaptive expertise with arithmetic refers to students’ ability to adaptively and flexibly use arithmetic strategies in solving mathematical tasks. There are only a few practical suggestions offered by research on how adaptivity with arithmetic problem solving can be developed. One general guideline is to provide students practice with different combinations of numbers and operations in order to develop a better understanding of the underlying numerical characteristics and relations in their problem-solving process (Baroody, 2003). The aim of the present study was to explore the affordances of the Number Navigation Game-based learning environment (NNG) in developing a richly connected mental representation of numbers which underlies and supports the development of adaptivity with arithmetic problem solving.

Literature Overview

Adaptive Number Knowledge

Adaptive and flexible arithmetic problem solving can be described by the ability to select the most “optimal” (fast and efficient) problem-solving methods for a given mathematical task. However, it is fairly relative what can be defined as “optimal” problem solving for a given problem, as this can depend on many factors such as problem characteristics, personal characteristics, and preferences, and also the norms and rules of a given social context (Verschaffel, Luwel, Torbeyns, & Van Dooren, 2009). There are several underlying factors to consider when exploring adaptivity with arithmetic problem solving. One such factor is the ability to notice and use numerical characteristics and relations when performing calculations on novel tasks where just recalling arithmetic facts is not sufficient. Based on the density and strength of numerical relations available for a person, different numerical connections can be noticed leading to various solution methods (Dowker, 1992). Thus, adaptivity with arithmetic problem solving can be described as involving the noticing and use of numerical characteristics and relations in order to arrive at efficient problem-solving strategies in mental calculations (Threlfall, 2002, 2009). Accordingly, efficiently noticing and using these connections requires a well-connected mental representation of numerical characteristics and relations. The knowledge of these characteristics and relations is a key component of adaptivity with arithmetic problem solving and is referred to as adaptive number knowledge in the present study.

The importance of numerical characteristics and relations has been explored in a number of studies on students’ adaptivity with arithmetic problem solving. Children with a well-connected representation of numbers use more flexible procedures in their mental computations (Heirdsfield & Cooper, 2004). Mathematical experts have been found to have more rich numerical connections and can flexibly use them in their mental problem solving (Dowker, 1992). Students who can understand the principles of commutativity or associativity are also more efficient and more flexible in their mental computations (Canobi, Reeve, & Pattison, 2003). However, even if students are able to report on a variety of procedures to solve a problem, this knowledge is often not used during problem solving, especially with novel problem types (Blöte et al., 2000; Canobi et al., 2003).

In previous studies, adaptivity with arithmetic problem solving was mainly measured using (a) measures of specific strategies, such as indirect addition and direct subtraction (e.g. Torbeyns, Ghesquière, & Verschaffel, 2009), (b) case studies, interviews, detailed analyses of problem-solving procedures (e.g. Heirdsfield & Cooper, 2004), or (c) language-intensive tests (e.g. Schneider, Rittle-Johnson, & Star, 2011). In this study, a measure of adaptive number knowledge is presented which aims to assess individual differences in students’ available mental connections of numerical characteristics and relations while solving novel types of arithmetic problems (McMullen, Brezovszky, Rodríguez Padilla, Pongsakdi, & Lehtinen, 2015). The Adaptive Number Knowledge Task is a timed paper-pencil measure that can be applied in a classroom setting. In this task, given a certain set of numbers, students need to look for various combinations of these numbers and the four arithmetic operations that would lead to a given outcome. The amount and complexity of different combinations provided by the students is considered to reflect students’ recognition and use of numerical connections in their problem-solving process, and thus represent a measure of students’ adaptive number knowledge.

Integrating Adaptive Number Knowledge Training and Game Mechanics

For developing stronger adaptive number knowledge, it is suggested, that more emphasis should be placed on providing students with contexts for discovering the underlying relations between numbers and operations. Working with various combinations of numbers and operations can aid the development of rich networks of numerical relations and help student to recognize and use these relations in their arithmetic problem solving (Baroody, 2003; Threlfall, 2002, 2009; Verschaffel et al., 2009). Game-based learning environments can provide an optimal context for discovery learning and exploration without the fear of failure (Devlin, 2011; Gee, 2003), which makes them promising tools for supporting extensive playful practice with various number–operation combinations.

Despite the educational potential of game-based learning environments, even recently published review studies report inconclusive evidence regarding the effectiveness of this medium (Girard, Ecalle, & Magnan, 2013; Wouters, van Nimwegen, van Oostendorp, & van der Spek, 2013; Young et al., 2012). This pattern is similar in the domain of mathematics instruction; although the literature suggests a growing trend in using games for developing different mathematical skills (Hwang & Wu, 2012), the number of empirically tested game-based learning environments is low and even published studies are often methodologically problematic (Cheung & Slavin, 2013; Heirdsfield & Cooper, 2004; Seo & Bryant, 2009). Additionally, many of the existing empirically tested game-based learning environments in mathematics are playful drill-and-practice training environments that aim to strengthening basic arithmetic skills, the results of which are hard to transfer in new contexts (e.g. Kucian et al., 2011; Räsänen, Salminen, Wilson, Aunio, & Dehaene, 2009; Wilson, Revkin, Cohen, Cohen, & Dehaene, 2006).

A further problem with existing game-based learning environments in mathematics is the inability to adequately integrate their core game mechanics and educational content (Devlin, 2011; Habgood & Ainsworth, 2011; Young et al., 2012). Game mechanics are crucial mechanisms through which players make choices and progress in the game (Salen & Zimmerman, 2004). The difference between an integrated and non-integrated game design is that in the first case, the player progresses through the game by doing math while in the second case the player is forced to do math in order to progress. Thus, when developing NNG used in the present study, the aim was to design a game which offers more than just the drill and practice of calculation fluency, but which aids the development of adaptive number knowledge through the core game mechanics where students work with number patterns and numerical relations.

The Relationship of Learning Goals and Game Performance

The NNG integrates the hundred-square representation of the natural number system with game mechanics in which players are required to use their knowledge of numerical characteristics and relations. This allows players to gain extensive practice with various number–operation connections. The core concept of the game is the external representation of base ten systems as a number square and the basic unit of the game is a map, which is a 10 × 10 number square superimposed over varying landscapes of sea and islands (see Fig. 1).

Fig. 1
figure 1

Example of an NNG map in the energy scoring mode (harbour at number 89, first target material at number 62)

Full size image

Because the aim of NNG is to go beyond the basic understanding of natural numbers, the hundred square was selected as the core representation of the number system (see a detailed theoretical rationale in Lehtinen et al., this volume). Compared to the linear number line, the hundred square provides students with a representation of the base ten system which better highlights the abstract, systemic nature of natural numbers. Thus, it is often used in activities where the aim is finding number patterns and numerical relations within the framework of natural numbers (Beishuizen, 1993). Additionally, recent research suggests that in addition to board games which use the number line as a basic representation (Siegler & Booth, 2004; Siegler & Ramani, 2009), using the 10 × 10 number square as the basis of a regular children’s board game also shows promising results in developing mathematical understanding (Laski & Siegler, 2014).

In NNG, within each map, the player has to navigate a ship in order to retrieve different target materials and build settlements, which means selecting certain combinations of numbers and operations as a result of which the ship will move from one location to the next (see Fig. 1). Within each map, players have to collect four types of target materials placed in different locations and return them to the starting harbour. Players need to choose their moves strategically in order to avoid islands, and adapt their strategies according to the two scoring modes of the game. With a total of 64 maps and a variety of rules and challenges, as well as different scoring modes, the game provides ample opportunities for working with various combinations of numbers and operations. For a more detailed description of the relation between different scoring modes and strategies used by the players, see the game design description by Lehtinen and colleagues (this volume) or Brezovszky, Lehtinen, McMullen, Rodriguez, and Veermans (2013).

In studying the effects of different learning environments such as game-based learning environments, one crucial issue is how students interpret the learning tasks and how they orient to the learning processes in these environments (Järvelä, Lehtinen, & Salonen, 2000; Lowyck, Lehtinen, & Elen, 2004). When analysing the effectiveness of learning environments, Engle and Conant (2002) have used the term “productive disciplinary engagement” to describe the kind of approach which results in productive learning. When discussing the educational effectiveness of a gaming environment, it is not enough that students are playing the game. Instead, what matters is how they focus on the core ideas of the learning tasks and the quality of their engagement in productive activities or productive learning behaviour (Chen, Liao, Cheng, Yeh, & Chan, 2012).

In NNG, the basic unit of the game is a map. Although players are free to access, close and return to any maps in any order, a meaningful game goal is to complete the maps by collecting the four target materials within each map. Only by completing maps can players collect the reward coins, which allow them to gain access to further maps and difficulty levels. Integrating the learning aims and the core game mechanics in a game-based learning environment is important as this method can aid students to meaningfully engage with the core learning aims (Devlin, 2011; Habgood & Ainsworth, 2011). When domain-specific content is integrated into the gameplay, as in the case of NNG, students’ game performance, or the amount of practice students have with the game, is a good representation of the amount of productive engagement with the targeted learning content. In the present study, the number of maps completed was considered as a measure of players’ game performance and a representation of their productive engagement with the learning content within the game context.

Aim of the Present Study

The aim of the present study is to explore if and how game performance is related to the development of adaptive number knowledge in primary school children using the NNG game-based learning environment. Providing students with environments to work with various combinations of numbers and operations is expected to be beneficial for developing their adaptivity with arithmetic problem solving (Baroody, 2003; Threlfall, 2002, 2009; Verschaffel et al., 2009). NNG provides vast opportunities for working strategically with various number patterns and number–operation combinations using an integrated game design where the learning gains and game performance should be highly related. Thus, we hypothesize that differences in students’ game performance in NNG predict the development of their adaptive number knowledge.

Method

Participants

Participants were 23 Finnish speaking sixth grade primary school students (11 female, M age = 12.2, age range: 11–13 years) from a single classroom. Students played NNG in pairs, though due to the odd number of students in the class one of them was using the game individually. The game data of the individual player was lost and one student was missing during post-test. Informed consents from parents and students were obtained before the start of the study. Ethical guidelines of the University of Turku were followed.

Procedure

A pre-test, intervention, and post-test design was used. Starting at the beginning of the autumn semester, students took part in a 12-week long intervention playing NNG for 8 consecutive weeks with a 1-week school holiday break. Pre-tests of mathematical skills were administered 2 weeks before the first playing session, and the post-tests were completed 2 weeks after the last playing session. Students played the game seven times in total, each Friday at school during math class. The whole 45 min long class period was afforded for playing. From these 45 min, the average time on actual playing the game during a class period was 26 min (SD = 2 min, 22 s), and average total time on task across the seven play sessions was 3 h and 3 min (SD = 18 min, 47 s). Play sessions took place in the classroom (three pairs) and in a computer lab just next to the classroom (eight pairs). Students only played the game during these 45 min sessions in class. The teacher and one to three teacher assistants were present during each playing session.

Students were playing in pairs, each pair having their own computer. The study did not impose any restrictions on the selection of pairs; the teacher was free to select pairs the way he wanted. More information on the player pairs is provided in Table 2 in the “Results” section. Pair play was chosen as the results of similar studies in game-based learning environments and mathematics suggest that collaborative play might lead to better attitudes towards mathematics and also better learning outcomes (Ke, 2008; Plass et al., 2013) and because a previous pilot study with NNG suggests that using the game collaboratively can enhance game-strategy-related discussion (Brezovszky et al., 2013).

Measures

Parallel versions of the paper-pencil measures of adaptive number knowledge (Adaptive Number Knowledge Task) and the same measure of arithmetic fluency (Woodcock-Johnson Math Fluency) were administered during pre- and post-test. Additionally, students’ game performance was saved in the game log data and students’ grades in mathematics from the previous semester were provided.

Adaptive Number Knowledge Task. The aim of the Adaptive Number Knowledge Task is to measure students’ ability to recognize and use different numerical characteristics and relations in their arithmetic problem solving. The test was developed based on the results of previously conducted pilot studies (McMullen et al., 2015). The task consists of four items and for each item students were given four or five numbers and a target number. Using the given numbers and the four arithmetic operations, the task was to produce as many solutions as they could which equalled the target number (see Fig. 2). The students had 90 s to complete each item. Tests were administered by a trained researcher during pre-test and by the class teacher with the help of a trained researcher during post-test.

Fig. 2
figure 2

Example item of the Adaptive Number Knowledge Task

Full size image

The format of the task was the same at both time points. Items were changed across time points but the given numbers and target numbers were selected so that similar types of number–operation combinations could be used in order to reach correct solutions. Table 1 shows the items in order for the two time points.

Table 1 Items of the Adaptive Number Knowledge Task during pre- and post-test
Full size table

Solutions were scored on the criteria of quantity and complexity of correct arithmetic sentences. Quantity was defined as the total number of correct solutions across all trials (correct solutions). Complexity was defined as the total number of solutions in which both additive and multiplicative operations were used (multi-operational solutions); for example, 6 + 4 + 2 = 12 or 8 + 8 − 4 = 12 was not considered to be multi-operational, but 2*3 + 6 = 12 was considered multi-operational.

Pre-test Cronbach’s alpha reliability scores were α = .55 for the number of correct solutions and α = .67 for the number of multi-operation solutions. Cronbach’s alpha reliability scores of all items were low on the post-test. Therefore, based on item analysis aimed at identifying a reliable uni-dimensional adaptive number knowledge score (Metsämuuronen, 2006), only the first two post-test items of the Adaptive Number Knowledge Task were used for analysis. For the post-test, the reliability score was α = .74 for correct solutions and α = .72 for multi-operation solutions.

Woodcock-Johnson Math Fluency. In order to measure students’ basic arithmetic skills, the Woodcock-Johnson Math Fluency sub-test (WJ III® Test of Achievement) was administered after the Adaptive Number Knowledge Task at both time points. The test was selected as it is a highly reliable and validated instrument measuring arithmetic fluency (Schrank, McGrew, & Woodcock, 2001). Following the original instructions, students had to complete as many arithmetic problems (simple addition, subtraction, and multiplication) as possible during 3 min. The test consists of two pages with a total of 160 items. Calculations become gradually more difficult as the test progresses.

Game performance. All data regarding players’ game activity was logged and saved. The number of maps completed was selected to be used as a measure of game performance in the present study. A map is completed when all the four target materials are collected within a map and the player receives a reward coin.

General math achievement. Students’ grades in mathematics from the previous semester were provided by the class teacher. Grades in the Finnish system range between 4 and 10.

Results

Detailed information on the composition of pairs, their gender, general math achievement (math grades), and game performance (maps completed) is presented in Table 2. Average sum scores for the number of total correct solutions and multi-operation solution on the Adaptive Number Knowledge Task, as well as students’ sum scores on the Math Fluency task for the two time points are presented in Table 3. With regard to overall game performance, out of the total 64 maps, students completed between 11 and 29 maps (M = 19.05, SD = 5.3).

Table 2 Gender, general math achievement, and game performance of the 11 player pairs
Full size table
Table 3 Descriptive statistics for the Adaptive Number Knowledge Task and the Woodcock-Johnson math fluency test at the two time points
Full size table

Independent samples t-test showed no gender differences in students’ game performance t(20) = 1.12, p = .28. Pearson product‐moment correlation showed no relation between students’ general math achievement and their game performance r(22) = .02, p = .918.

Overall, results show that there was substantial improvement in participants’ performance on the Adaptive Number Knowledge Task and Math Fluency measure during the period when NNG was used in the classroom. Paired samples t-test showed differences from pre- to post-test with medium to large effect sizes for the total number of correct solutions, t(21) = −7.97, p < .001, d = 1.49; total number of multi-operations used, t(21) = −2.72, p = .013, d = 0.55, as well as math fluency scores, t(21) = −4.10, p < .001, d = 0.46. Pearson product‐moment correlation was conducted in order to explore the relationship between students’ game performance and their adaptive number knowledge and arithmetic fluency (Table 4).

Table 4 Intercorrelations (Pearson product‐moment correlation) of adaptive number knowledge, Woodcock Johnson math fluency, and game performance
Full size table

In order to investigate the specific impact of game performance on the development of adaptive number knowledge multiple stepwise linear regression analyses were run. Two regressions were calculated, first with the number of correct solutions on the Adaptive Number Knowledge post-test as the dependent variable and second with the number of multi-operational solutions on the Adaptive Number Knowledge post-test as the dependent variable. For both regressions, participants’ pre-test scores of total correct and total multi-operational solutions and number of maps completed were entered stepwise as independent variables. No multicollinearity was detected for the three independent variables.

For total correct solutions, the final model was significant, F(1, 19) = 15.64, p = .001, R 2 = .45 with pre-test scores as the only significant predictor of post-test correct solutions (β = .67, p = .001). For total multi-operational solutions, the final model was fairly informative (Table 5), with 70 % of post-test scores being explained by participants’ pre-test correct and multi-operational solutions and the number of maps completed.

Table 5 Stepwise linear regression analysis: specific effects of multi-operation solutions pre-test, correct solutions pre-test and maps completed on multi-operation solutions post-test
Full size table

A third stepwise linear regression analysis was run in order to examine the possible impact of gameplay on the development of math fluency, with math fluency post-test scores as the dependent variable and math fluency pre-test scores and number of maps completed entered stepwise as independent variables. The final model was significant, F(1, 19) = 61.79, p = .001, R 2 = .77 with pre-test scores as the only significant predictor of math fluency post-test scores (β = .88, p < .001).

Discussion

The aim of the present study was to explore how students’ game performance in NNG predicts the development of their adaptive number knowledge. Results show that students’ game performance was a unique predictor of their post-test multi-operation solutions on the Adaptive Number Knowledge Task. Findings of the study are in line with the theoretical assumption that providing extensive practice with various combinations of numbers and operations can aid students’ noticing of numerical characteristics and relations, as indexed by their adaptive number knowledge (Baroody, 2003; Threlfall, 2002, 2009; Verschaffel et al., 2009).

From the point of view of the design of game-based learning environments, the results of the present study are promising. In order for a game-based learning environment to be efficient and effective players need to be engaged in productive activities which are relevant to the learning outcomes (Chen et al., 2012; Engle & Conant, 2002). Integrating the learning content and the core game mechanics is considered to help this process, resulting in higher engagement and better learning gains (Devlin, 2011; Habgood & Ainsworth, 2011). However, in practice it often happens that even if students are engaged with the game, they are engaged with aspects that are irrelevant to the educational content and learning goals (Garris, Ahlers, & Driskell, 2002; Martens, Gulikers, & Bastiaens, 2004). The relation between students’ game performance and their development in noticing and using complex numerical relations in their arithmetic problem solving suggests that working with the NNG could promote aspects relevant to the development of adaptive number knowledge. Thus, engaged gameplay in the NNG seems to be related to activities relevant to the intended learning goals.

The number of maps completed by the student pairs was selected as the basic indicator of students’ active engagement with the mathematical content in NNG. In order to complete maps and progress in the game students needed to be continuously working with various combinations of numbers and operations. These combinations needed to be strategically selected, taking into consideration the available numbers, target positions, and the different scoring modes of the game (moves or energy mode). Thus, measuring students’ game performance by the amount of maps completed during the seven playing sessions was sufficient for the goals of the present study. However, future studies could address alternative and more in-depth aspects of students’ game performance such as changes in players’ adaptive problem-solving procedures while playing the NNG.

Although the use of student pairs has pedagogical benefits (Hufferd-Ackles, Fuson, & Sherin, 2004), it also causes some problems for interpreting the findings. Differences in prior knowledge, gaming experience, or attitudes towards mathematics and games in general could all affect how students play in pairs, and how they share responsibilities during the game play. Thus, pair scores of game performance might not represent the true game performance of individual players and as a result, using pair game performance scores can be an underestimation of the effect of gameplay on the learning outcomes. Results should be confirmed with a substantially larger sample. Likewise, differences between playing in pairs versus playing individually need to be addressed in future studies.

In the present study, recognizing and using numerical characteristics and relations is described by the term adaptive number knowledge and is considered as a key component of adaptivity with arithmetic problem solving. The results of the present study suggest that NNG promotes the development of adaptive number knowledge. However, due to the small sample size and problems with the low reliability of the Adaptive Number Knowledge Task, these results have to be taken with caution. Although, students’ game performance explains variance in students’ post-test multi-operational solutions on the Adaptive Number Knowledge Task, the largest amount of variance was explained by students’ pre-test scores. In interpreting these results, it is also important to take into account that adaptive number knowledge was measured with far transfer tasks which were not directly practiced during the gameplay.

Results showed no relationship between students’ game performance and the development of their correct solutions on the Adaptive Number Knowledge Task. It is possible that playing NNG can only be associated with the task-specific development in students’ noticing and using complex numerical relations. As the Adaptive Number Knowledge Task is timed, it is possible that after playing the NNG students are more prone to invest effort in finding complex types of numerical relations; as a result, students would come up with fewer solutions overall compared to the pre-test. However, the lack of a relation between the number of correct solutions in the Adaptive Number Knowledge Task and students’ game performance may be explained by reliability issues and the small sample size. It is possible that by increasing the reliability of items, finding more appropriate indicators of adaptive number knowledge on these tasks (McMullen et al., 2015), and having a larger sample size, results would also show a relationship between students’ game performance and their development in the number of correct solutions on the Adaptive Number Knowledge Task.

Math fluency was used as a control measure in the present study. The lack of relation between students’ game performance and their math fluency is in line with the theoretical expectations that playing with NNG can be associated to a larger extent with the development of more adaptive aspects of arithmetic problem solving than number facts (e.g. Baroody, 2003). However, given the lack of statistical power in the present study, it is also possible that a significant relation could be identified in a larger sample.

There is an extensive theoretical framework describing the advantages of mathematics instruction which aims at developing adaptivity with arithmetic problem solving (Baroody, 2003; Blöte et al., 2000; Hatano & Oura, 2003; Threlfall, 2002, 2009). However, there are very few empirical studies which would offer practical guidelines on how this type of adaptivity could be operationalized and enhanced through classroom activities. Results of the present study suggest that the NNG is a promising game-based learning environment which can develop students’ adaptive number knowledge by enhancing noticing and using numerical characteristics and relations in arithmetic problem solving. NNG provides a novel type of game-based learning environment which offers more than just the drill and practice of already acquired mathematical skills, as students’ engagement with the core game mechanics is intrinsically connected to meaningful practice with skills and knowledge underlying adaptivity with arithmetic problem solving.