1 Introduction

Children and teenagers look as if they have an innate relationship with technology. They are exceptionally fluent in handling and using digital media and tools to such an extent that Prensky (2001a) refers to them as “digital natives”. In addition, the majority of them devote a remarkable portion of their spare time in playing digital games. Digital games play a key role in the entertainment of even for very young children, as the driving force behind their success as recreational activities are the fun and enjoyment they offer (Barlett et al. 2009; Zhang et al. 2010).

Coming to education, Mathematics is a teaching/learning subject that has a dominant place in primary school’s curriculum. However, the majority of students do not consider it as an easy subject. Due to the problems students face in understanding and using mathematical concepts and methods, they adopt a negative attitude towards this subject. The term math phobia aptly describes the fear and the insecurity they feel, which, in turn, has a negative effect on their performance (Brown et al. 1989). To overcome these problems and offer students opportunities for learning, there is the need to implement innovative teaching methods and to use state-of-the-art means/tools. For example, the role of the educator has to change; from the one who transmits knowledge has to turn into students’ coordinator and collaborator (Tolmie et al. 2010). Regarding the means that can be used, several of them are ICT tools, for example, tablets and/or specialized software for teaching mathematical concepts. Indeed, the relevant literature is quite extensive (e.g., Hong et al. 2000; Lieberman et al. 2009; Lowrie 2005; Zaranis et al. 2013). However, contrary to the compatibility of digital games with students’ mentality, and contrary to the assertion that they can be used in the teaching of several subjects (Mayo 2009), relatively few studies have been conducted regarding their use for teaching mathematical concepts (e.g., Boyle et al. 2016).

On the basis of the above observations, it was quite logical to wonder what the learning outcomes might be if digital games were used for teaching Mathematics to primary school students. For that matter, a project was designed and implemented. The target group was first, fourth, and sixth-grade students. The primary research question was whether (and to what extent) the use of digital games for teaching Mathematics in primary school can lead to better learning outcomes compared to conventional teaching methods. The research rationale and methodology, as well as the results and their implications, are presented and analyzed in the coming sections.

2 Mathematics and digital games

Ever since digital games came up, they have been considered as useful educational tools (e.g., Egenfeldt-Nielsen 2005; Prensky 2001a). Some even argue that they will soon be the predominant teaching tool (Johnson et al. 2012). Game-based learning (GBL), the idea that games can be used in everyday teaching (Prensky 2001b), began to apply to all levels of education and in most subjects. The supporters of GBL are exploring ways of integrating even commercial games in the educational practice, so as to achieve the learning objectives, to raise the interest and motivation for learning, and to create positive attitudes either towards specific subjects or towards the education as a whole (e.g., Bottino et al. 2007; Egenfeldt-Nielsen 2005; Ke 2008; Robertson and Miller 2009; Papastergiou 2009; Shaffer 2006; Squire and Barab 2004; Tüzün et al. 2009).

Digital educational games are considered to be highly effective in young people, the “digital natives” (Prensky 2001b). The catalyst to that is the fact that students are paying more attention to a learning activity when it occurs via a game (Garris et al. 2002). Also, it has been observed that when students play educational games, they tend to spend more time in trying to learn which, eventually, affects the learning outcomes (Sandberg et al. 2011; Tobias et al. 2011). Another feature of digital games is that they offer immediate feedback; students can swiftly see the results of their actions or if they answered a question correctly (Prensky 2001b). Also, students are encouraged to explore, experiment, and discover new concepts and strategies (Kirriemuir 2002). Digital games, in the long run, can positively influence the attitudes of students toward a learning subject (Hidi and Renninger 2006).

Coming to Mathematics, a number of digital games are, basically, drill and practice applications for the acquisition of very basic mathematical skills, such as automaticity in operation skills (e.g., Forlizzi et al. 2014; Miller and Robertson 2011; Polycarpou et al. 2010; Shin et al. 2012). On the other hand, there are games that rely on experiential learning, exploration, and experimentation, which help in understanding mathematical concepts, relationships, strategies, and norms (Kebritchi et al. 2010; Van Galen et al. 2009). Puzzle games seem to have a positive impact on computing skills, accuracy, and speed of carrying out arithmetic operations (Robertson and Miller 2009). Role-playing games, along with well-designed activities allow students to create links between abstract mathematical concepts and real-life situations (Shaffer 2006). A genre of games that are often used in Mathematics are mini-games, short games, easy to use, and focused on a very specific topic (Frazer et al. 2007; Jonker et al. 2009; Panagiotakopoulos 2011). Due to their usability, their short duration, and low hardware requirements, they can be played at flexible times and repeatedly, features useful in the educational use of games (Kebritchi 2010).

Research evidence suggested that by using digital games for teaching Mathematics, at primary school level and regarding the achievement of the learning objectives, the results are at least the same compared to teaching methods in which no games were used (Ke 2008; Rosas et al. 2003; Shaffer 2006). Researchers noted that games facilitated the creation and testing of hypotheses, as well as the promotion and development of critical thinking (Bottino et al. 2007; Lowrie 2005). In cases where games were used for quite a long time, a considerable increase in problem-solving skills was recorded (Bottino et al. 2007). Significant increase in motivation for learning and in students’ interest in Mathematics (Ke 2008; Robertson and Miller 2009; Rosas et al. 2003; Shaffer 2006), as well as better collaboration among students, and between students and teachers (Rosas et al. 2003; Robertson and Miller 2009) have also been reported.

Finally, in systematic literature reviews, out of a total of 259 empirical studies, from 2000 to 2014, only 16 were found to involve digital game for teaching concepts related to Mathematics (Boyle et al. 2016; Connolly et al. 2012; Hainey et al. 2016). In fact, some of them had either small samples (Bai et al. 2012) or there was no control group (Vogel et al. 2006). Thus, it can be argued that, although it is believed that digital games can help to overcome issues related to the teaching of Mathematics, as presented in the preceding paragraphs, there is enough room for further studies.

3 Research rationale, methodology, and implementation

As already mentioned, the purpose of the study was to examine the learning outcomes when digital games are used for teaching Mathematics to primary school students. The research hypotheses were:

  • H1. When using digital games the learning outcomes are better compared to other forms of instruction.

  • H2. Digital games attract the interest of students and motivate them to learn.

  • H3. Teachers can develop their own effective (in terms of the learning outcomes) educational games.

For examining these hypotheses, a project was designed and implemented. During its design stage, an invitation for participation was issued, addressed to teachers working in Athens, Greece. Most of the ones who responded affirmatively had to be excluded because their classes had too few students and it was not considered appropriate to combine students from different schools in order to create a “pseudo-school”, though others suggest it can be done (Ross 2005). In other cases, they were excluded because their schools were too far apart or because they were working in private schools and, consequently, the sample would not be homogeneous in terms of the socio-economic status of students (Institute of Education Sciences and National Science Foundation 2013). As a result, nine teachers were selected to participate in the project, working in public schools located in nearby districts of the city of Athens. To each, an instructional method, presented in the coming paragraphs, was randomly assigned. As a result of the above, the project’s target group was first, fourth, and sixth-grade primary school students (ages 6–7, 8–9, and 11–12). Students of the same grade were divided into one experimental and two control groups. In the first control group the teaching was conventional, while, in the second, a contemporary teaching method was used. The experimental group used digital games.

For conventional teaching, the school textbooks were used without any additional learning material and activities other than the ones included in the textbooks. Students did not work in groups; the teaching method was purely teacher-centered. Constructivism provided the theoretical framework for the contemporary teaching method. According to this theory, students construct their knowledge based on what they already know and by making links between new and old information, while their active participation in the entire process is essential (Ertmer and Newby 2013). Also, a key element in constructivism, but also in other contemporary views for learning, is collaboration (Tolmie et al. 2010). Consequently, the teaching method was based on the constructivist instructional model and specifically on the model proposed by Driver and Oldham (1986) which consists of five distinct phases:

  • Orientation, for motivating students towards the topic.

  • Elicitation, for assessing students’ prior knowledge and concepts.

  • Restructuring, in which students clarify and exchange their ideas and concepts with peers and teachers, and construct new ideas.

  • Application, where students test what they have learned.

  • Review, which provides students the time to reflect on what they have learned.

Students worked in pairs, were free to collaborate and discuss with each other, while teachers actively participated and guided them. The school textbooks were used, but also worksheets, designed by the teachers, were extensively used during the orientation, elicitation, restructuring, and review stages. The worksheets allowed students to familiarize themselves with the new learning material, formulate and write down their hypotheses, share ideas, expose them to cognitive conflict situations, test them, and provide explanations for the observed results. During the application stage, students solved real-life problems in order to relate what they have learned to their everyday lives.

As for the experimental groups, the teaching methodology included an orientation stage, a stage where students used the games, and a review stage. During the first and third stages, students worked in pairs, no games were used, the teachers actively participated and guided students, and the school textbooks together with worksheets were used as in the contemporary teaching method. On the other hand, the cognitive material as well as activities, problems, and exercises (corresponding to the restructuring and application stages of the previous method), were presented to students in-game. Students worked, once again, in pairs, were free to collaborate and discuss with each other. The teachers intervened only when there was a technical problem or a dispute between students; in essence, the teachers’ role was transferred to the games.

During meetings with the project’s teachers, four teaching units from each grade were selected. The selection criteria were (a) the units to be conceptually related and (b) to be relatively easy to convert them into games. For the first-grade, the units related to counting numbers up to 50 and then up to 100, as well as the numbering system (units and tens) were selected. For the fourth-grade units related to decimal numbers were selected (“Remember the decimal numbers”, money/weight/length and decimal numbers). As for the sixth-grade, the units were about converting decimal numbers to fractions, adding subtracting, and multiplying decimal numbers. Each unit, regardless of the teaching methodology, was taught in one two-hour session.

The games were developed using Microsoft’s Kodu Game Lab (http://www.kodugamelab.com/). Kodu is a programming environment designed exclusively for the rapid development of 3D games. The programming language, which is an icon/tile-based visual language, is very simple and even young adolescents can learn how to use it in a very short time. Furthermore, it provides ready-made cartoonish objects and characters, and a set of manipulation tools to build the games’ landscape. It has to be noted that the programming language is very close to the natural one. For example, it uses expressions like hear, see, bump, and combat, to trigger events and to implement interactions between objects and between objects and the user. On the negative side, it does not allow the import and use of external media (e.g., 3D models, images, videos, and sounds); the designer has to develop his/her games using only Kodu’s available media.

The games were not developed by a group of experts, but, instead, the participating teachers of the games groups were asked to do so. That is because it was considered crucial to examine how difficult it is, in real-life situations, for the educators to develop their own games/teaching material that will be used by their students. Consequently, the teachers had to follow the pipeline for developing games (e.g., come up with game scenarios, implement game mechanics, test the games, etc.). Since they had no previous experience in designing games and in using Kodu, they attended a 30-h intensive course for that matter. In addition, during the development of their games they had at their disposal printed and audiovisual material for guiding them. The only instructions that they received were based on Gee’s guidelines for designing “good” educational games (Gee 2009; 2005): players/students should be allowed to take advantage of the games, to provide simple control mechanisms, the cognitive material to be clearly presented, to provide compelling experiences for good learning, and to allow users/learners to enact their own trajectories. Furthermore, they were asked to find ways of presenting the learning material in-game, because, as it was mentioned in a preceding paragraph, this was a prerequisite of the games teaching methodology.

It has to be noted that teachers were able to come up with interesting game levels and scenarios. They were able of utilizing most of Kodu’s features and finding innovative ways to overcome the limitations imposed by the program’s limited number of objects and media (Fig. 1-4). On the other hand, all games were, essentially, drill and practice applications. These observations will be further elaborated in sections 4 and 5.

Fig. 1-4
figure 1

Screenshots from various games

Full size image

For collecting research data, the following were used: (a) pre-tests, to test the common cognitive starting point of all groups, (b) delayed post-tests, that were given about a week after the end of the interventions to check the sustainability of knowledge, and (c) evaluation sheets, administered immediately following the end of each session, in order to capture the immediate learning outcomes. The above tests, common to all groups, included, mainly, mathematical operations, right-wrong, multiple-choice, and fill-in-the-blanks questions. In addition, approximately a third of the questions involved problem-solving. A short questionnaire was improvised (15 Likert-type questions) that was given to the groups of students who used the games in order to record their views and attitudes towards them. Finally, teachers were asked to record the time that was needed for the development of their games, as well as the problems that they encountered during this process.

Prior to the beginning of the project, students’ parents were briefed, and their written consent for their children’s participation was obtained. Also, in order to proactively face any difficulties, students familiarized themselves with the controls of the games by playing mini-games, which were developed using Kodu, but without any educational objectives. One two-hour session proved to be enough because students, even the very young ones, did not encounter any problems in controlling and using the games.

4 Results analyses

While the initial sample size was 214 students, a number of them had to be excluded because they were absent in one or more session. The final sample size was 201 students, divided into nine groups (three grades X three teaching methods; Group0-conventional teaching, Group1-contemporary teaching, Group2-games). For the analysis of the results, scores were computed on the basis of the number of correct answers in each evaluation sheet (including pre- and delayed post-tests). Mean scores per group of participants and per test are presented in Table 1.

Table 1 Means and standard deviations on all evaluation sheets
Full size table

One-way ANOVA tests were to be conducted to compare the scores of the nine groups in all tests, in order to determine if they had any significant differences. Prior to conducting these tests, it was checked whether the assumptions of ANOVA testing were violated. Some minor issues regarding the normality of data were found in three cases. Like other parametric tests, the analysis of variance assumes that the data fit the normal distribution. On the other hand, literature suggests that ANOVA is robust to moderate deviations from normality (the absolute values of the skewness and kurtosis for the data not to be more than double their respective standard errors) and the false positive rate is not affected very much by this violation (Glass et al. 1972; Lix et al. 1996). Since in the above cases the violations were minor rather than moderate, they were considered as acceptable deviations. Since all the other assumptions were met (equal number of participants in all groups per grade, no outliers, and no violation of the homogeneity of variance), the analysis was conducted (Table 2).

Table 2 One-way ANOVA results
Full size table

Post-hoc comparisons were conducted using the Tuckey HSD test on all possible pairwise contrasts (except for the pre-tests because no statistically significant differences were noted there). The results of these tests are presented in Table 3.

Table 3 Post-hoc results
Full size table

Taken together, these results suggest that:

  • All groups had the same initial knowledge level since they did not have statistically significant differences in the pre-tests. As all groups had the same initial starting point, any differences observed in the participants’ knowledge acquisition after the interventions can be attributed to the different teaching methods that were followed.

  • Students in Group2 (games) outperformed students in Group0 (conventional teaching) in all fifteen cases.

  • Students in Group2 outperformed students in Group1 (contemporary teaching) in four cases, while the opposite applied in two cases. In nine cases the results were not statistically significantly different.

  • Students in Group1 outperformed students in Group0 in three cases, while in the rest twelve cases the results were not statistically significantly different.

The data analysis, as presented above, partially confirms the first research hypothesis. The learning outcomes when using digital educational games for teaching math concepts, are better compared to conventional teaching, but not that different compared to a contemporary teaching method.

Coming to the questionnaire that was given to students in the games groups, its purpose was to record their experiences and views regarding the games they played. From their answers, it is evident their strong positive attitude for the games (either for specific game elements of for the games as a whole) and the lack of notable problems. Moreover, students stated that they learned quite a lot (Table 4). Thus, the second research hypothesis was confirmed.

Table 4 Students’ questionnaire
Full size table

Though the examination of the games per se was not among the objectives of this study, it can be noted that: (a) all games were basically drill-and-practice applications, having exercises and activities similar to the ones included in the textbooks, and (b) in most cases, the teachers decided to offer the learning material in a large game level and develop several smaller levels/mini-games for additional activities, problems, and exercises. The former can be attributed to how teachers are used to conducting their lessons; offering the learning material first and then giving students exercises and problems for practicing what they have learned. The later can be attributed to a Kodu’s limitation, that of not allowing the development of large and complex game levels so that the hardware requirements for playing the games are kept minimal.

The games were very different from each other; thus, they cannot be easily compared, not even the ones developed by the same teacher. What is certain is that their development time was decreasing as teachers were getting more experienced in using Kodu (Table 5). The hours listed are the sum of the time needed from the initial idea to the finished product. As teachers pointed out, a considerable amount of time was dedicated to the refinement of their initial ideas and to the development of the games’ scenarios. Although they did not report severe problems regarding the programming of characters and objects, all three of them reported significant difficulties in implementing their games due to the limited number of objects available to them and because they were not able to import and use their own media. For example, a scale, as well as euros (coins and banknotes), were essential for the games that dealt with weight, money, and decimal numbers (fourth-grade), but they were not available. So, teachers had to come up with alternative and innovative ways of representing such objects, and this required additional time, thinking, and effort.

Table 5 Time needed for the game’s development (approximately)
Full size table

Finally, the games did not lack minor or major problems, both in terms of playability and of how well they presented the learning material. For example, in a mini-game, the user/student had to fetch objects (apples) representing the right answers to questions and bring them to certain spots. Alas, the apples were too far apart and too far from the right spots; students lost valuable time while searching for the correct answer and then bringing it to the right spot. As a result of the above, it is not easy to give a definite answer to the third research question and the issue will be further elaborated in the coming section.

5 Discussion

The main research hypothesis was that the use of digital educational games in math teaching can yield better learning outcomes compared to more conventional teaching methods. This teaching/learning subject was selected because of the difficulties students face in understanding and using mathematical concepts. Digital games were used because literature suggests that they can help students understand such concepts. For examining the significance of the findings, three different teaching methods were used and their results were compared.

Data analyses revealed that in all evaluation sheets (including the delayed post-tests), students in the games groups fared better than the ones who were taught conventionally. However, only in four cases, they outperformed the students that were taught with a contemporary teaching approach, the results were the same in most cases and, in two cases, students in the contemporary teaching groups outperformed students in the digital games groups. Quite interestingly, in most cases, students in the control groups did not have statistically significant differences with each other. Consequently, the first hypothesis can be partially accepted; in math teaching, the use of digital games produces better learning outcomes in comparison with conventional teaching, as suggested by previous studies (e.g., Brom et al. 2011; Ke 2008; Rosas et al. 2003; Shaffer 2006; Ulicsak and Wright 2010). However, the same does not -clearly- apply when comparing digital games and contemporary teaching, at least in the present study.

While a strict reading of the results leads to the above conclusion, there is another way of viewing and interpreting them. It can be argued that it is important that the use of digital games had -more or less- the same learning outcomes with a well-organized and contemporary teaching method. That is because the introduction of digital games in teaching, although something unprecedented for students, did not disorientate them and did not disturb the classes’ climate. Instead, on the basis of students’ responses to the relevant questions, a pleasant learning environment was developed, through which students achieved the same level of literacy compared to those who were taught using the constructivist instructional model.

Students, during the teaching phases where the games were used, worked on their own; the teachers’ guidance and involvement were kept minimal. The fact that students were able to perform well, seems to confirm the views of researchers who believe that students with a high degree of autonomy and increased control over their own learning process, can achieve good learning results (Hong et al. 2000; Mayer and Moreno 2003; Nunes et al. 2009). On the other hand, students worked in pairs and it seems that a satisfactory level of cooperation was achieved, at least according to their responses to the relevant question. Therefore, the views of researchers that support the notion that good results are achieved through teamwork (Cummins 2005; Tolmie et al. 2010), are also endorsed. Collaboration is cultivated through learning processes that involve the use of games (Gee 2008) because learners are actively engaged in experimentation, cooperation, and competition (Westera et al. 2008).

Fun seems to have played an important role in the achievement of the learning outcomes that were observed. Students’ fun and enjoyment when using games and their highly positive attitudes towards them are evident throughout the questionnaire. Fun and enjoyment probably motivated them for learning. Kebritchi et al. (2010), highlight the pleasant learning environment that is shaped when students are engaged with playing digital educational games, which, in turn, leads to increased incentives for learning (Binsubaih et al. 2006; Ke 2008; Robertson and Miller 2009; Prensky and Prensky 2007; Rosas et al. 2003; Shaffer 2006). It is worth noting that educational games motivate even students with learning disabilities (Annetta et al. 2009).

Students not only stated that they liked a lot the various game elements (e.g., music, game-characters, objects, and scenarios) but also that they prefer to play games and learn rather than read from textbooks. The appeal of digital educational games is identified in other studies (e.g., Papastergiou 2009) and it is a strong indication of how welcome this alternative way of teaching is.

Apparently, students did not face any serious problems when using the games and, in fact, they familiarized themselves quite quickly with their use, as it is evident in their responses to the relevant questions. Needless to say, that children are proficient users of gadgets and applications and can easily adapt to anything related to technology (Prensky 2001b; Whitton 2007) since this is compatible with their experiences and skills (Goodwin 2012; Heinrich 2012). Thus, it is logical that no problems were reported.

It should be noted that the time required for the development of the games was quite large. Although Kodu is not considered particularly difficult to learn, since it is addressed to children, the production of educational games, from someone who is not an expert in this field, proved to be a time-consuming process. Under certain circumstances, for example, for courses that have very specific learning objectives, such an effort might be unjustifiable, compared to the learning outcomes it yields (Kluge and Riley 2008). Therefore, there are implications for software engineers and designers. Given that teachers are not (and should not be) experts in computers and in programming, software engineers have to design user-friendly applications’ development tools. An even more important task is to redesign the entire pipeline of developing applications and to make it more flexible and intelligent in order to ease the process and reduce the production time (Scacchi 2012).

Finally, the time that was needed for teachers to develop their games, highlights an even greater problem. These games were the result of the work of amateurs; the teachers received just an intensive course on how to develop games, that could hardly render them adept game designers. One, exerting intense criticism, could characterize the games as incomplete, not correctly realized, or that they missed their learning objectives. To some extent, this criticism is acceptable and, indeed, it is quite possible that the learning outcomes were adversely affected by the games’ minor or major shortfalls. However, at least in Greece, similar educational applications, developed by professionals and certified for their educational worthiness, simply do not exist. Despite countless studies that demonstrated the significant educational benefits when using ICT applications in various teaching subjects, teachers still continue to flounder alone without support from policy makers and education administrators. Sporadic research efforts or the goodwill (and hard labor) of individual teachers are not going to have any meaningful effect on a lethargic and resistant to changes educational system. Substantial reforms in policy and planning are needed for rectifying the situation. On the positive side, it can be argued that when an amateurish attempt was able to produce “good” results, professionally made games are expected to have even better ones.

6 Conclusion

The study’s key finding was that students in the games groups grasped the mathematical concepts that they were taught better than the ones that were taught conventionally and, in some cases, they outperformed the students that were taught using a contemporary teaching method. Their increased interest and motivation when using the games were also noted.

Although the results are interesting, the study has limitations that must be taken into account. The sample, although sufficient for statistical analysis, was fairly limited both numerically and geographically. It is therefore quite difficult to generalize the results. Relatively few mathematical concepts were examined. More concepts would have allowed an in-depth examination of the problem. The use of more and diverse research tools, for example, interviews and observations, would have allowed more detailed research data. The games that the teachers were able to develop were not examined per se; certain aspects of them could have been better. Finally, students may not have been completely honest in their responses, confusing the questionnaire with some form of evaluation. Future studies can develop and use games that cover a larger portion of the mathematics syllabus of all primary school’s grades. This is achievable, if not for all, certainly for several modules. It would also be interesting to conduct research maximizing or minimizing the teacher’s role, and/or by using other types of applications and/or devices (e.g., tablets) and compare the results. By doing so, it would be possible to determine if the outcomes can be attributed to the medium used and/or to the method.

In conclusion, the need for changing the way Mathematics are taught in primary school and the disengagement from conventional teaching approaches is almost self-evident. Digital games offer an interesting alternative method, worthy of further investigation.