Background and introduction

An educational app is a web-based or downloadable software application that typically works on a mobile device and is designed to support learning (Bouck et al. 2016; Papadakis et al. 2017). A number of studies have suggested that mathematics apps used in primary school classrooms have a significant positive impact on learning performance (Bulut et al. 2015; Moyer-Packenham and Suh 2012; Riconscente 2013; Thambi and Eu 2013). However, the projected number of apps available in 2017, both free and paid, is over 5 million, 15% of which focus on education (Statista 2017; Technavio 2015). In other words, roughly 750,000 apps are available in the domain of education. It is not surprising, then, that many educators find the selection of appropriate apps to be a monumental task (Alon et al. 2015; Bouck et al. 2016; Papadakis et al. 2017). Several researchers have attempted to address this selection challenge by developing metrics to classify mathematics apps (Alon et al. 2015; Ebner 2015; Grandgenett et al. 2011; Handal et al. 2016). Collectively, these classification schemes have proposed at least four different types of mathematics apps: practice-based (Alon et al. 2015; Grandgenett et al. 2011; Handal et al. 2016; Ebner 2015), constructive (Alon et al. 2015; Grandgenett et al. 2011; Handal et al. 2016), productive (Alon et al. 2015; Grandgenett et al. 2011; Handal et al. 2016) and game-based (Ebner 2015). To date, these classification schemes are based on limited theoretical and empirical evidence. The purpose of this study is to conduct an exploratory analysis and comparison of four types of mathematics apps (practice, constructive, productive and game-based) based on three characteristics (perceived learning value, usability, engagement).

Literature review

Overview

A review of the literature was conducted based on peer-reviewed articles published from 2005 to 2017 focusing on the use of mathematics apps in primary school classrooms. Three themes are discussed based on the proposed research questions: impact of mathematics apps on learning performance, type of mathematics apps and essential characteristics of mathematics apps.

Learning performance

Research exploring the impact of mathematics apps on learning performance in primary school classrooms has produced mixed results. With respect to practice-based mathematics apps, Burns et al. (2012) reported that grade 3 and 4 student performance was significantly higher than matching control groups. On the other hand, Nurmi and Jaakkola (2006) observed that learning performance for grade 4 and 5 students who used practice-based apps was not significantly different from control groups. Regarding constructive apps, four pre-post studies without control groups indicated that primary school students learning performance improved significantly with effect sizes ranging from 0.34 to 1.26 (Kong and Kwok 2005; Moyer-Packenham and Suh 2012; Reimer and Moyer 2005; Suh and Moyer 2007). In addition, two quasi-experimental studies reported that treatment groups using constructive apps significantly outperformed control groups with effect sizes ranging from 0.84 to 1.27 (Bulut et al. 2015; Thambi and Eu 2013). One study reported no significant difference between pre- and post-test for grade 3 students using constructive apps to learn mathematics procedures (Reimer and Moyer 2005). Only one study assessed the impact of game-based apps on learning performance (Riconscente 2013). After controlling for fractions knowledge, liking and self-efficacy, primary school students using the Motion Math game-based app group performed significantly better than the control group, although the effect sizes were moderate (Riconscente 2013).

To date, no studies have explored the impact of productive apps on learning performance. Furthermore, no research has been conducted comparing different types of mathematics apps for primary school students. The disparity in learning performance results might be easier to understand if more information was provided about the type and characteristics of mathematics apps used.

Type of mathematics apps

A review of the research classifying mathematics apps collectively revealed at least four types of applications that would be appropriate for primary school students including practice-based, constructive, productive and game-based (Alon et al. 2015; Ebner 2015; Grandgenett et al. 2011; Handal et al. 2016). Other apps types focusing on problem-solving (Jerzembek and Murphy 2013), collaboration (Ebner 2015) and metacognitive skills (Ellis et al. 2012) have not been examined in the realm of mathematics and may be too advanced for primary school students.

Practice-based

Practice-based apps are designed to help students learn content and apply specific mathematics skills (Alon et al. 2015; Grandgenett et al. 2011; Handal et al. 2016; Ebner 2015). Typically, a range of questions are posed for a specific topic, and students receive some form of immediate feedback guiding their learning. This type of app supports and targets the learning of basic facts and concepts (Alon et al. 2015; Grandgenett et al. 2011; Handal et al. 2016; Ebner 2015). Examples of this type of app are BrainPop, Khan Academy and Memrise. While mathematics teachers are encouraged to promote critical and reflective thinking (NCTM 2000), basic knowledge of content and concepts is required to engage in higher levels of thinking (Ambrose et al. 2010; Donovan et al. 2000; Schunk 2008). Practice-based apps, then, are used to support the acquisition of foundational mathematics knowledge (Hirsh-Pasek et al. 2015). Several researchers have investigated the effectiveness of this type of app in primary school classrooms (Burns et al. 2012; Nurmi and Jaakkola 2006, Zhang et al. 2015). All three studies reported that grade 3 and 4 students significantly improved in short-term learning after using practice-based mathematics apps.

Constructive

Constructive apps focus on exploration (Handal et al. 2016; Murray and Olcese 2011), making sense of new information (Grandgenett et al. 2011), skill acquisition and data management (Domingo and Gargante 2016) and the active manipulation of ideas and concepts (Keenwge 2013). This type of app could be particularly useful for extending and applying basic content and conceptual knowledge. Examples of constructive apps are Desmos, Geogebra and Gizmos. The theoretical grounding of this app type rests in discovery learning (Bruner 2009) and social constructivism (Vygotsky 1978). Numerous studies in Canada (Kay and Knaack 2008), Finland (Nurmi and Jaakkola 2006), Malaysia (Thambi and Eu 2013), Taiwan (Chang et al. 2013), Turkey (Bulut et al. 2015), and the United States (Moyer-Packenham et al. 2013; Suh and Moyer 2007; Watts et al. 2016) have shown that constructive apps have a significant impact on learning for primary school students.

Productive

Productive or tool-based apps (Murray and Olcese 2011) are used to produce artefacts or create products (Grandgenett et al. 2011; Handal et al. 2016). Typical, productive apps would be used as a culminating activity to demonstrate and apply key knowledge, concepts and understanding (Prensky 2010). The use of productive apps is grounded in project-based learning theory (Kokotsaki et al. 2016; Thomas 2000). Examples of productive apps are Google Sheets, Piktochart and Show Me. None of the 18 research papers reviewed in this study examined the use of productive apps in primary school classrooms.

Game-based

Game-based apps involve learning and practicing concepts while playing games (Ebner 2015; Kiili et al. 2014; Riconscente 2013). In a typical education-based game, students are exposed to challenging activities wrapped within a narrative and structured with rules, goals, progression and rewards (Whitton 2014). Often, educational content is artificially injected into a game creating a quiz like atmosphere with extrinsic rewards (Riconscente 2013). Examples of game-based apps include Minecraft, Lure of the Labyrinth and Prodigy. One advantage of a game-based app is regular interaction and engagement (Kiili et al. 2014). Another benefit is that game-based apps with a strong narrative can deeply engage students (Hirsh-Pasek et al. 2015). A qualitative study with 153 primary students indicated that a game-based mathematics app, Math Elements, stimulated engagement in learning (Kiili et al. 2014). Another study reported that mathematics confidence and knowledge of grade 4 students increased when using game-based iPad apps (Riconscente 2013). Game-based apps can inhibit learning when students spend considerable time experimenting with deliberately incorrect responses to receive engaging and visually stimulating feedback (Falloon 2013).

Characteristics of apps

A review of the literature assessing the quality of mathematics apps at the primary school level revealed three prominent themes: perceived learning value, engagement and usability.

Perceived learning value

The primary goal of any app is to support and promote effective learning. Not surprisingly, then, one of the most researched characteristics of mathematics apps is perceived learning value, often measured by assessing student perceptions. Riconscente (2013) reported that 95% of grade 4 students agreed that the mathematics app, Motion Math, helped them learn. Kay and Knaack (2008) observed that younger students were relatively neutral about the learning value of the mathematics apps they used; however, the students appreciated the visual supports. Freebody et al. (2007), in a large-scale study of students from Australia and New Zealand, noted that students were neutral about whether mathematics apps helped them learn, although they did like working at their own pace, getting immediate feedback and receiving supportive prompts. Finally, Reimer and Moyer (2005) indicated that only half of the grade 3 students felt that mathematics apps helped them understand fractions better.

Usability

Usability or how easy it is to use a mathematics app is an important characteristic to consider for younger students. Directing cognitive load toward using an app instead of acquiring a concept will likely inhibit learning. For example, if the language level were too challenging, especially for students just learning how to read, the usability of an app could be impeded significantly (Ebner 2015). Additionally, if the content level were increased too quickly, students could get lost and disengage from mathematics apps altogether (Falloon 2014). Freebody et al. (2007) noted that students, on average, agreed that mathematics apps were easy to use and that they did not need help from their teacher to use them. Reimer and Moyer (2005) reported that 15 out of 19 grade 3 students rated apps as easy to use and faster than pencil and paper. Kay and Knaack (2008) also noted that a majority of middle school students rated mathematics apps as being easy to use. Finally, a review of the literature by Clark and Luckin (2013) observed that students viewed iPads, and by extension apps, as easy to use. Understanding variances in usability for different types of apps will help educators determine which of these tools may be appropriate for younger students.

Engagement

Engagement is also considered an important characteristic of mathematics apps (Freebody et al. 2007; Kay and Knaack 2008; Reimer and Moyer 2005; Riconscente 2013; The Learning Foundation 2009). Kay and Knaack (2008) reported that elementary school students were relatively neutral about the engagement value of the mathematics apps they used. Freebody et al. (2007) and the Learning Foundation (2009) noted that primary school students from Australia and New Zealand were enthusiastic about mathematics apps finding them interesting and fun to use. Interview data from Reimer and Moyer’s (2005) study indicated that grade 3 students enjoyed using mathematics apps. Finally, Riconscente (2013) observed that 100% of grade 3 students wanted to use the Motion Math app again. However, Fredricks et al. (2004) have suggested that engagement is more than simply liking an app or wanting to use it again. They propose that engagement consists of at least three components: behaviour (involved in activities), emotion (positive and negative reactions) and cognition (investment in learning). Mathematics apps, then, might be emotionally engaging and highly interactive, for example, but they also need to be cognitively engaging to support learning (Cayton-Hodges et al. 2015; Hirsh-Pasek et al. 2015).

Purpose

The purpose of the current study was to investigate the use mathematics apps from the perspective of primary school students. Specifically, two research questions were addressed:

  1. 1.

    What is the impact of using mathematics apps on primary school student’s learning performance?

  2. 2.

    How do four different types of mathematics apps (game-based, practice-based, constructive and productive) compare based on three app characteristics (perceived learning value, usability, engagement)?

Method

Participants

Twenty elementary school students, between seven and nine years of age (11 males, nine females), enrolled in a grade 2/3 split class, participated in the study. One student was on an Individualised Education Plan (IEP), and there were no English Language Learners. The students lived in a city in Southern Ontario, Canada, with a population of about one million people. The average family income for parents in the school district area was approximately $150,000 Canadian dollars. The school population was about 400 students with 6% classified as English Language Learners and 16% with special needs (Fraser Institute 2016).

Teaching context

The teacher in this study had taught in a primary school classroom for 4 years. She was comfortable and positive about using technology in her mathematics class. The classroom was equipped with a teacher’s laptop, an LCD projector, a smart document camera and Internet access. Tablets could be booked for use in the classroom when required.

For a typical class, the teacher divided the students into grade-specific groups, assigned an independent task to one group, then taught a lesson to the other group. The teacher used a variety of teaching tools including chart paper to share understanding, manipulatives such as beads to explore the concepts of addition and subtraction and worksheets for students to practice skills. Students spent 15–30 min each day working individually with the assigned mathematics app as directed by the teacher. Students practiced basic addition and subtraction questions with the practice-based app (Math Tappers). Students would work on addition and subtraction word problems, with the constructive app (Thinking Blocks) assisting them in developing a visual model. Students created video explanations with the productive app (Show Me) to demonstrate and communicate their understanding of subtraction and addition. Finally, students practiced and reviewed their addition and subtraction knowledge in a microworld environment with the practice-based app (Prodigy).

Research design

The primary goal of this exploratory study was to investigate and compare four types of mathematics apps based on three app characteristics. Additionally, the overall impact of mathematics apps on short-term learning performance was assessed. Both quantitative (surveys and test scores) and qualitative (open-ended questions and interviews) data were collected and triangulated (Maxwell 2005) to form a more comprehensive understanding of how mathematics apps affected student attitudes and learning. An explanatory mixed-methods approach was used where quantitative results were further explained by qualitative data (Creswell 2014).

Data collection

Three data collection tools were used in this study: surveys, interviews and tests. Survey data (Likert and open-ended questions) were used to examine student perceptions of specific mathematics app characteristics. Interview data were collected to augment and expand upon survey responses. Pre- and post-tests were delivered to measure learning performance over a five-week period.

Survey (demographic data)

The first three survey questions identified student gender, grade level and year of birth. Demographic data were limited to three variables to keep student identity anonymous (Appendix A—Items 1–3).

Survey (attitudes)

A thorough review of literature collectively revealed three essential mathematics app characteristics including perceived learning value, usability and engagement (Cayton-Hodges et al. 2015; Freebody et al. 2007; Grandgenett et al. 2011; Handal et al. 2016; Kay and Knaack 2008; Reimer and Moyer 2005; Riconscente 2013). Consequently, a five-point, Likert-scale was used to measure student attitudes about the four mathematics apps they used focussing on these three characteristics for each of the four apps used. De Leeuw (2005) noted that asking questions of young children can be challenging, therefore survey items in this study were pilot tested, kept to a minimum and written to be simple and easily understood. (Appendix A—Items 4–7). Two open-ended questions were added to assess what students liked and disliked about using the mathematics apps (Appendix A—Items 8–9).

Interviews

Six randomly selected students agreed to participate in an interview, and consent was obtained from their parents. These students were asked about what they explored, whether the use of iPads was helpful and whether they experienced any problems Appendix B—Items 1–4). Each audio-recorded interview took about 10–15 min to complete.

Pre- and post-tests (performance)

Pre- and post-tests were administered at the beginning and end of the study and assessed grade 2 or grade 3 addition and subtraction skills. The test focused on number comparison (2 questions), subtraction (6 questions), addition (6 questions), finding missing numbers (3 questions), estimation (4 questions) and word problems (2 questions). See Appendix C for a sample test.

Mathematics apps

Several steps were followed to select the mathematics apps for this study. First, a technology lead-teacher was consulted and recommended several apps that were used regularly at the school. In addition, key education resources were consulted including Teachers with Apps, Education World and Smart Apps for Kids. Finally, two university professors from the faculty of education reviewed the final mathematics apps. Full descriptions and locations of the four apps selected are presented in Table 1.

Table 1 Description and Evaluation of Mathematics Apps
Full size table

Procedure

In week one, the practice (Math Tappers) app was introduced first as it focused on basic skills for one and two-digit addition and subtraction. In week 3, the constructive (Thinking Blocks) app was used to help students learn addition and subtraction word problems. In week 4, students were encouraged to review and demonstrate their addition and subtraction skills by creating videos using the productive (Show Me App). Finally, in week 5, students reviewed their addition and subtractions skills using the game-based (Prodigy) app. The step-by-step procedure is outlined in Table 2.

Table 2 Overview of the procedure
Full size table

Data analysis

Descriptive statistics and frequency analyses were used from the Likert-scale questions to assess student perceptions of learning value, usability and engagement for each mathematics app. A content analysis (Stemler 2001) was conducted for the open-ended responses on the survey and interview questions to elaborate on the characteristics assessed by the survey. Finally, pre- and post-tests were conducted using a paired t-test.

Results

Learning performance

Pre- and post-tests were conducted to assess students’ performance levels before and after using all five mathematics apps. A paired t-test revealed that the mean learning performance scores were significantly higher after the use of mathematics apps (M = 75.8, SD= 21.5) than before (M = 59.7, SD= 20.1) (t (19) = 3.1, p < 0.005, d = 0.80. According to Cohen (1988 1992), the difference between means is considered large based on the Cohen’s d value of 0.80.

Perceived learning value

Table 3 provides a comparison of primary school students’ attitudes about the perceived learning value of mathematics apps examined in this study. The game-based app (Prodigy) was rated the highest with almost 95% of students agreeing that it helped them learn to add or subtract. Open-ended and interview responses were consistent with these ratings:

  • “The Prodigy helped me to add and subtract better.”

  • “It helped me more on adding and subtracting.”

  • “Many questions were on adding and subtracting, and it helped me learn through playing games.”

Table 3 Student ratings of perceived learning value of mathematics apps (n = 20)
Full size table

The constructive app (Thinking Blocks) received the second highest ratings with 70% of students agreeing that it helped them learn (Table 3). Students’ comments supported these ratings:

  • “[Thinking Blocks] helped me to learn addition and subtraction.”

  • “Thinking Blocks taught me how to solve word problems faster, and I got better at it after using the app.”

  • “I liked the word problems [in the Thinking Blocks app], and I liked when I had to decide if I have to do addition or subtraction for the question. It helped to solve the word problems with blocks”.

The productive app (Show Me) received similar ratings to the constructive app (Prodigy) with respect to perceived learning value with 65% of the students agreeing that it helped them learn (Table 3). However, only one comment was offered about learning support for this app, “I like recording better than talking in front of the class, and it helped me to solve word problems.”

Finally, the practice-based app (Math Tappers) was rated the lowest in terms of perceived learning with 50% agreeing that it helped them learn (Table 3). Student comments, though, revealed that some students thought the app was useful:

  • “It helped to add higher numbers.”

  • “[The] apples helped to learn subtraction better.”

Usability

Table 4 presents a comparison of primary school students’ attitudes about the usability of mathematics apps. All of the students agreed that the practice-based app (Math Tappers) was easy to use. Student comments supported these ratings:

  • “It was easy to use because I could tap the numbers and I didn’t have to type it.”

  • “It was simple, not complicated.”

Table 4 Student ratings of the usability of mathematics apps (n = 20)
Full size table

The game-based app (Prodigy) was rated second highest in terms of usability, with over 80% of students agreeing that it was easy to use (Table 4). However, students did not comment on this usability of this app.The constructive app (Thinking Blocks) was considered relatively difficult to use with only 45% of primary school students rating it high in terms of usability (Table 4). Student comments reflected these low ratings:

  • “It was hard to figure out what to do.”

  • “Sometimes when I pressed a number, it didn’t show up.”

  • “Sometimes the box wouldn’t be dragged from one place to another, and the steps were not clear.”

  • “Thinking Blocks was the hardest app to use… the questions were given to me, and I had to solve the questions. Also, I find it hard to use because there [was] a lot of information on the screen”.

Finally, the productive app (Show Me) was considered the most difficult to use with less than one-quarter of the students rating it high in terms of usability (Table 4). Students’ comments aligned with these low ratings:

  • “It was hard to find the letters on the screen keyboard.”

  • “It was complicated and little hard to do.”

  • “[This app] was challenging because if you type the wrong thing, and if I go back, all the work is erased.”

Engagement

Table 5 compares student ratings of engagement among the mathematics apps used in this study. All students agreed that the game-based app (Prodigy) was engaging. However, only one comment was offered, “I liked how you can choose your own character, and it was fun.”

Table 5 Student ratings of engagement for mathematics apps (n = 20)
Full size table

Ninety-percent of the students agreed that the practice-based app (Math Tappers) was engaging and their comments reflected this:

  • “It was fun to practice, and I got better at it.”

  • “I liked that it was fun because it was very well designed and interesting to use.”

  • “It was fun because you got to do more challenges and you can buy stuff with your rewards. It just felt I like was in the game.”

  • “Math Tapper was fun and gave me actual numbers to add to solve the question.”

Sixty-five percent of the students agreed that the constructive app (Thinking Blocks) was engaging. Sample comments were

  • “It was fun doing adding and subtracting with the Thinking Blocks app.”

  • “I liked it because it helped me to learn better and it was fun.”

  • “The Think Blocks App was fun to use.”

Finally, almost 60% of the students agreed that the productive app (Show Me) was fun to use. No comments were offered by students about the engagement value of this app.

Summary of app comparison

The game-based app (Prodigy) was the mostly highly rated in terms of usability, perception of learning and engagement. The constructive (Thinking Blocks) and productive (Show Me) apps were rated the second highest in terms of learning, but lowest in terms of usability and engagement. The practice app (Math Tappers) was rated highest in terms of usability and engagement, and lowest in terms of learning (Table 6).

Table 6 Comparison of mathematics apps (n = 20)
Full size table

Discussion

Learning performance

Ultimately, the goal of integrating mathematics apps into the classroom is to improve learning. Therefore, it was important to establish whether the apps examined in this study had some impact on performance. Based on the paired t-test, primary school students’ short-term learning performance improved significantly after using the four mathematics apps. This finding is consistent with previous research on academic performance after using a variety of mathematics apps (Burns et al. 2012; Kong and Kwok 2005; Moyer-Packenham and Suh 2012; Nurmi and Jaakkola 2006; Reimer and Moyer 2005; Suh and Moyer 2007). However, other factors such as the quality of the teacher’s instruction, the support that students received from their parents at home, the use of physical manipulatives and worksheets most likely influenced the positive test results. Therefore, it is unreasonable to suggest that the mathematics apps were the only factor that enhanced student learning performance. Mathematics apps in this study were integrated within a system of learning tools and resources to augment the teaching and learning process, not replace it. We can assume, though, based on the survey responses, as well as the open-ended and interview data that most students believed the mathematics apps contributed to their learning. The next question was to address the relative contributions of the four mathematics apps types to the learning process.

Comparing types of mathematics apps

Perceived learning value

Previous studies have reported that primary school students are neutral about the learning value of mathematics apps (Freebody et al. 2007; Kay and Knaack 2008). The results of the current study suggest that perceived learning value is partially dependent on the type of mathematics app used.

The game-based app (Prodigy) was universally perceived as the most helpful in terms of learning basic subtraction and addition concepts. The app was used to review concepts already attained, so it is interesting that students rated it as most helpful with respect to learning. It is possible that high ratings for usability and engagement influenced student’s perceptions of learning. In addition, the game-based app could have been played at home and this extra time on task, while not formally documented, may have contributed to perceptions of increased learning.

The constructive (Thinking Blocks) and productive apps (Show Me) were considered helpful in terms of learning by about 70% of the students. Attributions of better learning could not be attributed to usability or engagement, as these two apps were rated the lowest for these two characteristics. Both the constructive and productive apps appeared to require more cognitive effort, and students, even at this young age, seemed to appreciate the challenge and ultimate learning benefits. This explanation is consistent with previous studies advocating the importance and need for cognitive engagement when using apps (Cayton-Hodges et al. 2015; Hirsh-Pasek et al. 2015)

The practice app (Math Tappers), rated relatively high for usability and engagement, was rated lowest in perceived learning value. This result may not be surprising given that the main purpose of the app was to practice concepts that students already knew. While it may be satisfying to demonstrate understanding by getting simple questions correct, many students appeared not to perceive this activity as new learning. Practice-apps that address newly learned content or skills would likely receive higher ratings of perceived learning.

It is worth noting that both open-ended and interview comments about learning were quite general, perhaps to be expected from students in grades 2 and 3, and did not provide in-depth insight into student ratings. Further research, possibly in the form of more structured interviews, focus groups and observations are needed to explore mathematics app learning benefits

Usability

Previous research suggested that primary school students view mathematics apps as being easy to use (Freebody et al. 2007; Kay and Knaack 2008; Reimer and Moyer 2005). In this study, ratings for usability varied considerably depending on the type of app used. All students agreed that the practice app (Math Tapper) was easy to use, possibly because it had the most straightforward design. In addition, the concepts covered were basic and cognitive overload, a factor that could contribute to lower usability ratings, would be unlikely.

Just over 80% of the students agreed that the game-based app (Prodigy) was easy to use. It is not surprising that this app was rated as harder to use than the practice app, given that it had a complex design involving different challenge levels, high interactivity, rich graphics and a wide selection of problems. The app’s game-like, story-based structure may have enhanced engagement and motivation, thereby explaining why it was perceived as being easier to use than the Thinking Blocks and Show Me apps.

Less than 50% of students agreed that the constructive app (Thinking Blocks) was easy to use. Three factors may have contributed to these ratings. First, students were attempting to solve word problems which were considered to be the most cognitively challenging part of the unit. Second, the design was not intuitive, and the instructions were minimal. Third, the excessive amount of text on the screen, especially for primary school students, is likely to have been difficult to read. All three factors would likely increase the cognitive load of students and therefore decrease ratings of usability.

Only 25% of the students rated the productivity app (Show Me) as easy to use. Unlike the other three apps, the design was open-ended, the creation of videos was an entirely new task and quick rewards or feedback were limited. This app, on average, may have been too difficult for this age group to use in the one-week time period allotted.

Engagement

Several studies have suggested that mathematics apps are engaging to use (Freebody et al. 2007; Reimer and Moyer 2005; Riconscente 2013; The Learning Foundation 2009). Similar to the perceived learning value and usability characteristics, engagement ratings varied based on the type of app used.

Every student rated the game-based (Prodigy) app as being fun to use. Prodigy offered a strong narrative, clear goals, a distinct rate of progression, high interactivity, adaptability to student responses and rewards. These factors have been reported as highly desirable in past studies (Kiili et al. 2014; Riconscente 2013; Whitton 2014).

Ninety-percent of the students agreed that the practice-based app (Math Tappers) was fun to use noting that they liked the play-like, visual learning environment and the feeling of getting better at doing problems. Apps with a strong visual component appear to attract the attention and interest of primary school students.

Only two-thirds of the students rated the constructive (Thinking Block) app as fun to use. As stated earlier, the focus on challenging word problems with limited visual representations and excessive text may have contributed to lower ratings. Students seem to view this app as cognitively but not emotionally engaging. Finally, just over half the students agreed that the productivity app (Show Me) was engaging. This app may have been perceived as less fun by primary school students because it was hard to use, completely different from the other three apps and void of any extrinsic rewards.

Fredricks et al. (2004) proposed that there were at least three types of engagement (behavioural, emotional and cognitive); however, they have not been examined with respect to mathematics apps to date. Based on open-ended comments and interview responses, primary school students mentioned and appreciated cognitive engagement extensively. They particularly enjoyed getting better, learning how to add and subtract and being able to solve problems. Further research is needed on the relative contributions of behavioural, emotional, and cognitive engagement.

Educational implications

Primary school students rated the game-based mathematics app (Prodigy) high in terms of perceived learning value, usability, engagement, challenge-level and feedback. This app provided a sufficiently challenging learning environment, positive feedback and rewards, friendly competition among students, adjusted questions based on student skill-level and constructive feedback. Game-based mathematics apps may be a promising choice with respect to supporting the learning of primary school students. However, as Riconscente (2013) suggests, teachers need to consider the quality of the game and richness of interactions and avoid game-based apps designed primarily for entertainment, not learning (Riconscente 2013).

Many students rated the constructive mathematics app (Thinking Blocks) relatively high in terms of perceived learning value and engagement, but found it difficult to use. A number of students appreciated being challenged by this tool but at times found the feedback to be too detailed. It is worthwhile noting that the topic addressed by this app, word problems, is difficult for primary school students, therefore characteristics like usability and engagement may be attenuated. All characteristics do not have to be rated high for a mathematics app to be successful in terms of learning. More research, though, is needed to determine the extent to which usability might interfere with engagement and learning.

The productivity app (Show Me) received ratings similar to the constructive app—it was quite challenging to use, somewhat engaging and provided a clear learning benefit. The design of this app, like many productivity apps, was open-ended and younger students may require more scaffolding, structure and time to use this type of tool.

The practice-based app (Math Tappers) received top ratings on usability and engagement; however, a majority of the students felt that the tool did not help them learn. It is important to recognise that easy to use, entertaining mathematics apps may not necessarily lead to better learning. However, teachers need to carefully consider learning goals and ability level when selecting practice-based apps. This type of app may be best suited to reviewing newly learned concepts and procedures. Furthermore, practice-based apps that adapt to a student’s ability level might provide more a differentiated learning experience.

Limitations and future research

There are a number of limitations of the methodology used in the current study that would help guide future research. First, app characteristics need to be examined in more depth. For example, overall perceived learning value could be expanded beyond basic content and extended to higher-level activities such as searching for information, reflecting, problem-solving and collaboration. It would also be helpful to broaden the scope of the engagement characteristic to behavioural, emotional and cognitive sub-categories, as suggested by Fredrick et al. (2004). To achieve this level analysis, observational or video recordings of students using mathematics apps could be used, perhaps with the experimenter probing the student about what they are thinking, doing and/or feeling. In addition, more app characteristics might be considered in the selection process. Promising criteria that are yet to be examined for mathematics apps include quality of feedback (Cayton-Hodges et al. 2015; Falloon 2013; Hirsh-Pasek et al. 2015), differentiation (An et al. 2015; Cayton-Hodges et al. 2015; Milman et al. 2014), learning goals (Hirsh-Pasek et al. 2015; Keengwe 2013), quality and authenticity of mathematics content (Alon et al. 2015; Cayton-Hodges et al. 2015; Moyer-Packenham et al. 2008) and support for collaboration (Handal et al. 2016; Hirsh-Pasek et al. 2015; Keengwe 2013).

Second, larger scale studies focusing on a wider selection of mathematics apps, with students from a variety of backgrounds and grade levels will help to determine the generalizability of results. Furthermore, focusing on learning over a longer time period would be more robust in terms of investigating the impact of mathematics apps.

Finally, no research could be found examining pedagogical choices made by teachers when using mathematics apps. It is argued that these choices are critical for app use to be successful. To some extent, the type of app selected should be tailored to the specific learning outcomes. For example, a teacher needs to consider whether students need to practice, articulate, reflect or construct. Various app characteristics like usability, challenge level and learning goals also need to be considered when integrating mathematics apps for students with different ability levels.