Keywords

1 Introduction

This chapter presents a serious game developed for the basic mathematics course, which is an area where STEM students have the most difficulty. In addition, information about the usefulness of the game is provided. The chapter continues with a general introduction, literature review, method findings, discussion, and, finally, conclusions. It is thought that both practitioners and researchers will find the chapter useful.

Over the past few decades, serious games have been used in different topic areas within education. Many empirical studies on serious games as an educational tool have yielded positive results. Although there is no single and definitive definition of serious games, researchers generally define them as games in which the main goal has an educational objective rather than being merely for entertainment. With the popularity of mobile devices, serious games have begun to move onto this platform. This chapter presents the development of a serious game for use in the field of mathematics education, a STEM field, and testing of the usability of the game.

With the widespread use of phones and tablets, the use of mobile devices in the field of education has the potential to meet students’ needs. Wartella et al. (2018) point out that in the USA, children under the age of 8 use tablets and smartphones. Mobile devices are often preferred by children due to being cheap and lightweight and having touch screens. The key features of these devices are touch screens, mobility and design, interaction through motion, accessibility, connectivity, and ease of acquisition (Fernández-López et al., 2013). The game presented here was developed in such as way as to take advantage of the features provided by mobile devices.

Despite an increasing national and international focus on science, technology, engineering, and mathematics (STEM) education, K-12 students continue to struggle with STEM content, resulting in very few students being successful in STEM fields, especially science and mathematics. Understanding effective and purposeful teaching and assessment strategies can help teachers provide effective teaching for a wide range of learners (Basham & Marino, 2013). Unfortunately, in PISA results, 50% of students in 24 countries fell below the minimum level of proficiency in mathematics (OECD, 2019). Mathematics is one of the most difficult subjects for students in the STEM disciplines (Topçu & Yıldız Durak, 2019). Serious games can be used to overcome these difficulties (Kiili et al., 2015). One of the main reasons for using games in educational settings is that students find them more interesting and engaging than traditional learning environments (Torbeyns et al., 2015). In addition, games can provide a more active learning experience than the normal classroom setting. The study reported here developed a serious game for use in math education in an attempt to overcome some the problems students face in this topic area. In this chapter, we present the development of the game and the test of its usability.

Educational games can evoke engagement and motivation (Kalogiannakis et al., 2021; Westera, 2019), facilitate student learning (Wouters et al., 2013), and enhance students’ problem-solving skills (Sánchez & Olivares, 2011). Games have been developed for most STEM fields. In a review study, Boyle et al. (2016) point out that knowledge acquisition is the main outcome in educational games. They improve student learning and motivation in the domain of STEM. Games can be a tool for facilitating the teaching of science, mathematics, technology, and engineering, i.e. STEM disciplines (Smith, 2020). Serious games may be used to enable students to automate the four processes in math (Fokides, 2018). There are serious games in the field of STEM, especially for students at the beginning of primary school, for example, Monkey Tales and Zeldenrust games (Torbeyns et al., 2015). As the early development of math skills is vital for students’ next steps, the game developed in this study focused on 2nd graders in primary school.

For serious games to be effective in supporting learning, games must be appropriately usable. It is important that serious games for educational environments are well designed and have a good level of usability, and care was taken to address these two main features in the development of the game. At this point, the concept of usability should be defined.

Usability is significant in games. With effective menu, settings, and controls, the aesthetics and mechanics of game design is not essential, but it does facilitate smooth playing of the game. Games need to be useable so that the learner can easily master the game controls and focus on the content (Olsen et al., 2011). It is important to examine usability through basic features such as efficiency, playability, and ease of use (Smeddinck, 2016). According to the three-tiered approach put forward by Olsen et al. (2011), it is important to evaluate games not only in terms of general usability but also in terms of playability and learnability. The ISO 9241-11 usability components are effectiveness, efficiency, and satisfaction (Bevan et al., 2016).

One of the most accepted usability evaluation criteria was put forward by Nielsen (2012). Nielsen describes usability as having five components: learnability, efficiency, memorability, errors, and satisfaction. Studies have been carried out using Nielsen’s (2012) components. Zaki et al. (2017), for example, examined the usability of the therapeutic game ASAH-I, looking at learnability, efficiency, errors, and satisfaction. Although they identified various problems, its usability was found to be generally suitable. Hussain et al. (2014) tested the usability of JFakih Learning Game for children aged 9–15 using nine criteria, including Nielsen's components. Although they found the game useful and attractive, they also identified areas that needed improvement. Using Nielsen's components, Almeida et al. (2019) examined the usability of the game ALTRIRAS, which was developed for students with autism and recorded generally positive results. Saman et al. (2019) carried out a usability test on a serious game with eight children with hearing loss, considering effectiveness, efficiency, and user satisfaction. They found it had a usability rating of 91.89%. Ismail et al. (2011) studied the usefulness of the game Jelajah with five pre-school children according to effectiveness, efficiency, and satisfaction. Their results showed an overall usability level of 73%. Mahdi (2017) measured the usability of a game developed to teach mathematics to children according to effectiveness, efficacy, and satisfaction and found the game usable. This current study uses Nielsen’s (2012) five components to measure usability.

Usability studies have been made for various games. However, more research is needed regarding well-designed serious games in STEM education. The aim of this study is to determine the usability of a mobile serious game developed according to design principles for a 2nd grade primary school mathematics course.

2 Serious Games

The literature review includes serious games, classification of serious games, the benefits of serious games, mobile serious games, serious games for mathematics, and research in this area. Under this title, general information about serious games and the place of serious games in mathematics from STEM fields are discussed.

Serious games do not have a single and precise definition in the literature and various definitions have been put forward. In general, serious games help the user to reach a desired objective while being entertained.

Prensky (2001) states that games can contribute to young learners’ learning processes. The term serious games was first coined by Clark Abt, author of Serious Games (1970), who used war games and simulations in his studies for developing curricula and training. Serious games are often intended for learning (Abt, 1970) and while there is no universally accepted definition of serious games, they are generally accepted as digital games that have at least one purpose in addition to entertainment (Dörner et al., ), which Dörner et al. refer to as “characterizing goals” (Dörner et al., 2016b). For example, if a game aims to teach mathematical concepts besides being entertaining, then it can be called a serious game. Educational games, a subset of serious games, cover many areas from kindergarten to university, individual and collaborative learning, special education, vocational education or on-the-job training, and health games that address mental and physical health (Dörner et al., 2016a). Dörner et al. (2016a) suggest serious games are generally designed for learning in different subjects, for example, mathematics (Barros et al., 2019), science (Baek et al., 2016), and special education (Durkin et al., 2015).

It can be seen from the literature that serious games with potential for education have been developed for different educational levels, in different genres, for different purposes, and for different target audiences. In this context, the varieties and classifications mentioned are also increasing. These details will not be included in this study.

3 Mobile Serious Games

The features of mobile devices, such as being accessible anywhere and anytime, being personalized, allowing students to learn at their own pace, and allowing easy and fast communication with other people during the learning process, make them suitable in learning environments (Gocheva et al., 2020). Due to their rapidly advancing functionality, mobile devices are frequently used in learning environments (Crompton et al., 2017). Research has shown that students between 8 and 12 years old spend an average of 1 h 17 min per day playing mobile games (Rideout & Robb, 2019).

Sharples and Pea (2014) emphasize that since ancient times, people have learned from the external environment. Mobile devices help to reinforce such learning by enablıng students to learn independently of time and place. With devices being used everywhere, we can say that there are advantages in mobility.

Mobile games, on the other hand, are games defined by the platform. Today, mobile devices that are continually being develop now have the technical capacity to run many mobile games, making them very popular (Laato et al., 2020). The mix of serious games and mobile learning provides advantages for using mobile devices in learning environments (Yallihep & Kutlu, 2020).

The rapid arrival of mobile devices in society has resulted in their convenience for learning environments. At this point, the independence of time and place comes to the fore in the literature, and an emphasis is placed on portability. Lightweight portables are essential devices for serious gaming, given their battery life.

4 Serious Games for Math

Mathematics is a fundamental discipline and a vital skill the must be acquired for students to succeed in today's society. Individuals who fail in basic mathematics can experience problems in their professional life. Unfortunately, many students in primary and secondary school experience failure and disappointment in mathematics (Huang et al., 2014). However, such weaknesses can be overcome by offering students different approaches, and games provide important support for mathematical development (Kiili et al., 2015). Serious games offer a different alternative in math learning environments and may contribute to students overcoming difficulties in mathematics.

Serious games in mathematics teaching can increase student engagement and motivation and facilitate their learning (Barros et al., 2019). Bakhuys Roozeboom et al. (2017) revealed that serious games lead to higher quality learning and that they have more positive effects compared to the traditional classroom environment. In their review, Hainey et al. (2016) looked at games developed for the field of science. The result of Tokac et al.’s (2019) meta analysis indicate that math learning improves better with games than in the traditional environment. Because of these advantages, serious games can play an important role in improving mathematics learning and students’ attitude to mathematics.

Educational games with a teaching purpose as well as being entertaining can be designed to teach mathematics. Such games can work on different devices, such as phones, tablets, and computers, and offer teachers various options for use in teaching mathematics (Pope & Mangram, 2015) (See Fig. 20.1). In fact, teachers’ attitudes are very important in the use of such educational applications (Poultsakis et al., 2021). Similarly, parental support is also of undeniable importance (Vaiopoulou et al., 2021). Encouraging teachers and parents to use and encourage the use of these games is an important dimension in using games in mathematics teaching.

Fig. 20.1
figure 1

Subsets of serious games (Laato et al., 2020)

Full size image

4.1 Available Research on Serious Games for Math

Many math games have been developed to support student learning and to be fun (Yıldız Durak, 2019). Duffy et al. (2017) developed serious games in arithmetic and geometry for primary school students, subjects that form the basis of STEM education. It was found that test scores were higher for students who played games. Pope and Mangram (2015) developed a game called Wuzzit Trouble, which aimed to improve 3rd grade students’ number sense. Of the 59 students in their study, they noted a significant difference in favor of those who played the game over those who did not. van der Ven et al. (2017) tested a mobile game covering addition and subtraction on 103 1st grade students. The calculation efficiency of the students who played the game was found to be high. In Fokides’s (2018) study, 201 1st, 4th, and 6th grade primary school students played games that teach basic math skills. The study found that the groups that played games understood the subject better and had increased motivation and interest. The results of empirical studies indicate benefits both academically and in terms of motivation and attitude. Given these results, the study reported here aimed to design and develop a serious game for mathematics lessons.

Hung et al. (2014) tested a game aimed at teaching line symmetry figures on 69 5th grade students. The authors state that it had a positive effect on students’ learning, motivation, and self-efficacy in mathematics. Brezovszky et al. (2019) tested the Number Navigation game on 1168 students from 5 to 7th grade. A significant difference was found in favor of the experimental group in terms of adaptive number knowledge and math fluency. Kyriakides et al. (2016) taught algebra to 15 primary school students (10–11 years old) with the mobile game A.L.E.X. They concluded that the students worked willingly and developed positive relationships with mathematics. In their study, Chang et al. (2016) had 107 5th grade students play a game aimed at teaching fractions. The results show that the students’ level of participation increased. Rodríguez‐Aflecht et al. (2018) tested the game Number Navigation on 212 5th grade students. Although some negative results were recorded over time, they state that most of their students were motivated while playing. The results of these studies are generally positive, particularly on factors such as students’ learning, motivation, self-efficacy, and participation, suggesting the use of serious games has promising results.

Robust research on instructional design features to increase the effectiveness of games in learning is scarce (DeLeeuw & Mayer, 2011). As a matter of fact, Papadakis (2021) emphasize that low-quality mobile applications are more common than scientific-based ones. However, the number of studies investigating quality, well-designed serious mobile games is still insufficient. We therefore developed a serious game for math scientifically based on the rigorous pillars of instructional design in an attempt to add to the evidence of the usability of well-designed games. The aim of the study is to reveal the usability of the mobile serious game developed according to design principles for a primary school 2nd grade mathematics course.

5 Goal—Purpose of the Research

This research aims to design, develop, and test the usability of a mobile game for primary school students to be used in mathematics education. For this purpose, a 2D mobile game was developed and Unity and usability tests were conducted.

6 Method

6.1 Research Model

The study used the evaluation criteria put forward by Nielsen (2012) to evaluate the usability of the developed game. Nielsen describes five components of usability: learnability, efficiency, memorability, errors, and satisfaction. Learnability refers to the ease with which users do tasks they encounter for the first time. While evaluating this criterion, the tendency of the users to use the game, the number of correct transactions in their first use, and the time spent using it are taken into consideration (Solmaz-Evcil & İslim, 2012). This study determined that the level of difficulty users who encounter the game for the first time face while performing the given tasks and the less support they receive from outside, the better the learnability of the game. Efficiency is the speed with which the user performs the task once they have learnt it, and memorizability is how well the user can use the system again after not using it for a while. When evaluating this criterion, situations such as the time to perform the task, the tasks performed per unit time, the number of aids used, the time spent on aid, and the effort expended are taken into account. The adaptation processes the users go through were examined while they performed the tasks provided in the game. It was determined that the better the speed of adaptation to the game, the better the efficiency of the game. Errors are those made by the user that are solved, and satisfaction is the degree of pleasure the user experiences while using the game. The fewer errors there are in the game, the more confidence the user has in using it (Abrahão et al., 2008). In this study, the errors that occurred while performing the tasks given in the game were observed and at the end of the time given for performing the tasks, users were asked whether they would like to try again and whether or not they liked it. As a result of the observations and the answers given, the error and satisfaction status of the game was evaluated.

Usability research was used in the study. Data obtained through different methods increases the validity and reliability of the results (Yıldırım & Şimşek, 2005). Qualitative methods aim to reveal events in their natural environment, holistically and realistically, using data collection tools such as observation, interview, and document analysis (Yıldırım, 1999). With quantitative methods, phenomena and events are measured and expressed numerically (Büyüköztürk et al., 2008). For this study, participants were provided with the educational game and observed playing it without intervention, using their own technological tools in their home environment. Users are only supported when necessary. The whole process was recorded through Zoom. The participants were also asked whether they liked the game or not, and their opinions were taken into consideration. The participants were evaluated in their performance of the predetermined tasks by examining the records.

6.2 Participants

For the educational game, an easily accessible sample group of primary school students was studied. Bevan (2006) recommends eight to ten participants in order to identify all usability problems. The sample in the current study consisted of 10 primary school students, seven girls and three boys. The educational level of the students in the sample were as follows: three primary school 2nd graders, two primary school 3rd graders, three primary school 4th graders, and two secondary school 5th graders. Of the students in the sample, one was seven years old, two were eight years old, two were nine years old, two were ten years old, and three were 11 years old. Information about the students who took part in the usability test is presented in Table 20.1.

Table 20.1 Participant information
Full size table

6.3 Data Collection Tool

Usability tests can be considered as one of the most efficient usability methods recommended for carrying out with real users. According to Nielsen (1993), when testing an interface, it provides real information based on how users use it and what problems they encounter while using it. This usability test was developed by the researchers. Five tasks were used in this test, which cover the mathematics lesson activities presented in the developed game environment. In determining the tasks, the opinions of a mathematics teacher and a field expert experienced in game development were taken. While evaluating the tasks in the usability test, successful and unsuccessful criteria were taken into account. Any user who fulfilled a task completely was considered successful in the task. If the user was not able to perform the requested tasks within the optimum time frame, they were considered unsuccessful. In addition, in the evaluation of usability, the average time spent by the users on the tasks was determined, and the status of the tasks within this optimum time was interpreted. Five tasks were determined by the researchers for usability testing:

  1. 1.

    View the help page.

  2. 2.

    Complete the first level in 90 s.

  3. 3.

    Complete the second level in 90 s.

  4. 4.

    Complete the third level in 120 s.

  5. 5.

    Complete the fourth level in 120 s

Participants were given 60 s for Task 1, 90 s for tasks 2 and 3, and 120 s for tasks 4 and 5. If the participants could not complete the task within the specified time, the task was deemed unsuccessful.

6.4 Instrument—Procedure

The first stage in the game development process was to devise a game development design plan. Then, in line with the game scenario, the graphics and visuals were planned and the design phase was carried out using Photoshop. The Unity 2018 2D application was used in this study.

For creating the scene plan and designing and arranging the visuals and objects, color and lighting processes suitable for the age level of the targeted player group were used as much as possible.

In the Unity program, the graphics and objects prepared by adjusting the scene design, camera, and plane positions are placed on the screen. The design process of the game is as follows (Figs. 20.2 and 20.3).

Fig. 20.2
figure 2

Game flow chart

Full size image
Fig. 20.3
figure 3

Game work flow chart

Full size image

Four Operations Mathematics Game Design Plan

  1. 1.

    Game overview

  1. 1.1

    Game concept

This game is a single player 2D space balloon defense game. The player uses their mathematical knowledge to stop the balloons coming to earth from space. The game was developed with the Unity game engine. It can be played on the web and on Android and IOS operating systems.

  1. 1.2

    Game overview

The game aims to develop players’ basic mathematical skills and to enable them to practice mental arithmetic operations in a fun way. Aimed at children of primary school age, it provides practice in mathematical operations in the form of a fun game as an alternative to pen and paper exercises. As the pace of the game increases, players must do the operations faster.

  1. 1.3

    Target group

This game is designed for players over seven years old.

  1. 1.4

    Game flow

The player must burst balloons descending from space and aiming to hit the world. The player must fire a weapon at the level of the balloon. To do this, the player must calculate a mathematical operation under the guns as quickly as possible and enter it into the input box. When they enter the correct answer, the gun fires and the balloon becomes ineffective.

  1. 1.5

    Learning objectives

    • Students will be able to perform mental addition.

    • Students be able to perform mental subtraction.

    • Students will be able to perform mental multiplication.

    • Students will be able to perform mental division.

  1. 2.

    Game mechanics

  1. 2.1

    Gameplay

During the game, players provide the necessary commands using a mouse and keyboard. Players select the weapon they want to use with the mouse and enter the answer using the keyboard. If the action response is wrong, the gun gives a warning and does not fire. When the action answer is correct, the fired weapon moves toward the balloon. If the player can block 100 balloons, they win the game. If five balloons hit Earth, the game is lost.

  1. 2.2

    Game items

Balloons: They start falling from a random column at the top of the screen at a random time.

Missile Batteries: This is the area at the bottom of the screen where the operation is asked and the gun is fired.

Missiles: The missile moves from the arsenal to the balloon if the operation is correct.

Health Bar: As balloons hit Earth, the health bar value decreases. When the 5th balloon hits, the health bar is reset.

Time Indicator: This shows the length of time the game has been played and the speed of the game increases at certain time intervals.

Score Indicator: This shows the number of balloons blocked.

Sound Effects and Music: When a weapon is fired, sound plays as the missile moves. When the missile hits the target, an explosion occurs.

Help System: The player is informed about the gameplay game on the login screen. When the operations are carried out incorrectly, the help window opens and the player is supported.

  1. 2.3

    Game and mechanics

Playing the Game: During the game, all movements are performed using the mouse and keyboard. The player uses the weapon systems by performing mathematical operations (addition, multiplication, subtraction, and division). A new target appears when a target is hit. Throughout the game, the main character is displayed on the screen in the top perspective.

Win: A game stage is considered won if the player manages to hit the required number of targets within the game time.

Lose: If the player cannot prevent five targets from hitting Earth, the stage is not completed and the game ends.

Movement: A two-dimensional one-way movement method is used in the game. The duration and speed of the movement are related to the player's response time.

Game Elements-Time: The game requires players to complete actions within a certain amount of time. Otherwise, the player loses the game.

2D environments: The game contains objects such as balloons and weapon systems in the space environment.

Maths: The game contains areas in which the player answers questions involving the four basic operations.

  1. 3.

    Technical descriptions

  1. 3.1

    Target hardware

    • A computer with standard hardware and the Unity Player plug-in or codec pack installed.

    • Standard keyboard and mouse.

    • For mobile devices with Android 6 or IOS 8 and above installed.

  2. 3.2

    Development software

    • The game is designed using Unity 2017.41f.

    • In the preparation of the game, the standard Unity Assests and other necessary assets were downloaded and included in the game library.

    • The code block of the game was created with C# programming language and Visual Studio 2017.

    • Photoshop CS6 was used for graphics and effects.

    • The AfterEfect and Illustrator programs were used for animations and animation effects in the game.

Objects for the game, for example UI objects and prefabs that ensure fluency and continuity, were created and embedded in the scene. The time and position settings of these created objects were made and the motion and animation phases were started. The fictional interaction between the objects was created by designing the relation network between animations and effects. Following the design phase, the code blocks for animating events, flow, and fiction were written, and association processes between the objects that make up the game and the code blocks were established. Once the coding process was complete, necessary adjustments (linear rendering mode, etc.) were made to convert the final output to the most suitable format for different screen sizes. The game was prepared for use on Android and IOS operating systems.

6.5 Applying Usability Testing

The usability test was carried out during online meetings with participants held on Zoom. The tasks were given to the participants in order and they were asked to complete a subsequent task on completion of the current task. The researchers did not intervene while the participants were completing the tasks. Finally, the participants were asked their opinions about the game. The meetings were recorded. After application, the video recordings were examined by the researchers and the data was analyzed.

7 Results

The usability test of the game was applied online to 10 primary school students. Students were asked to complete five tasks in order. The status resulting from the usability test applied online is indicated in Table 20.2 as “+” for those who completed the test and “−” for those who did not.

Table 20.2 Participant task completion information
Full size table

The results show that the participants read the help page, Task 1, within 60 s. The 1st level addition, Task 2, was completed by the 4th and 5th grade students and the K10 2nd grade student, while the students in other grades and the 5th grade K5 student could not do the addition.

Only the 4th grade students could do the 2nd level multiplication, Task 3. However, two 5th grade students (K3, F5) could not do it either.

Only the 4th grade students and the 2nd grade K10 student could do the 3rd level addition, Task 4. However, two 5th grade students (K3, F5) could not do it either.

Only the 4th grade students K8 and K9 could do the 4th level multiplication, Task 5. However, two 5th grade students (K3, F5) could not do it either.

Table 20.3 shows that Task 1 was successfully performed by all participants. The students who successfully completed the task did so in an average of 6.8 s.

Table 20.3 Task 1
Full size table

Table 20.4 shows that five participants could not complete Task 2 in the assigned 90-s period. The participants K3, K7, K8, and K9, who successfully completed the task, are 4th and 5th grade students and K2 is a 2nd grade student. Successful participants completed the task in an average of 57.4 s.

Table 20.4 Task 2
Full size table

Table 20.5 shows that five participants could not complete Task 3 in the assigned 90-s period. In fact, two participants skipped the task, stating they did not yet know the multiplication process. K7, K8, and K9 students, who are 4th graders, successfully completed the task. These participants completed the task in an average of 68.3 s.

Table 20.5 Task 3
Full size table

Table 20.6 shows that six participants could not complete Task 4 in the assigned 120 s. All of the 4th grade students (K7, K8, and K9) and the 2nd grade student (K10) successfully completed the task in an average of 105.7 s.

Table 20.6 Task 4
Full size table

Table 20.7 shows that six participants could not complete Task 5 in the assigned 120 s. As in the third task, two participants skipped the task because they did not know the multiplication process. Only K7 and K8 students were able to successfully complete this task. Those students who successfully completed the task did so in an average of 105 s.

Table 20.7 Task 5
Full size table

In this study, the Mann–Whitney U test was conducted to determine differences between user task times and successful task completion by gender.

Table 20.8 shows that no significant differences were found between the task performance of male and female users from Task 1 to Task 5. Although there is no statistically significant difference, the descriptive statistics show that the mean rank of male and female students are equal regarding achievement of Task 1. For Task 2, Task 3, and Task 5, the male student average is higher, while in Task 4, the female student average is higher.

Table 20.8 Mann–Whitney U test results of task performance by gender
Full size table

Table 20.9 shows no significant differences were found between the task performance periods of male and female users from Task 1 to Task 5. Although no statistically significant difference could be found, the descriptive statistics show that the time performing Task 1, Task 2, and Task 4 was lower for male students than female students. Female students performed Task 3 and Task 5 faster.

Table 20.9 Mann–Whitney U test results of task duration by gender
Full size table

8 Discussion-Conclusion and Recommendations

The aim of this study was to design, develop, and test the usability of a mobile game for primary school students for use in mathematics education. For this purpose, a 2D mobile game was developed with Unity and usability tests conducted with 10 primary school students. Usability and user experience are important measures of the quality of software. In the field of education, these quality characteristics are necessary to ensure an appropriate teaching process (Salas et al., 2019; Yildiz Durak, 2021). In order to provide a positive gaming experience for users, a game must be usable (Law & Sun, 2012).

According to Nielsen (2012), usability is a quality feature that evaluates the ease of use of application interfaces. It is defined by five components: learnability, efficiency, memorability, errors, and satisfaction.

Learnability: This can be expressed as the ease of performing basic tasks when users first encounter the design (Nielsen, 2012). Olsen et al. (2011) suggest that insufficient usability in games negatively affects students' learning.

Looking at the game designed for this study in terms of learnability, although participants acheived a 100% success rate for Task 1, their success rate for the other four tasks did not exceed 50%. It can therefore be concluded that the learnability of our educational game is weak. The average learnability of the game in their study was 3.6 out of 5. The reason was that the children were not familiar with the game at their first attempt. They point out that the children got accustomed to it afterwards. Zaki et al. (2017) reveal that more than half of the games they looked at were completed in a short time by users and that learnability was generally good. In contrast, the learnability score was high in Almeida et al.’s (2019) game study. Their observations suggest that this situation was mostly caused by insufficient time being given to the users for the game tasks. It is expected that the success rate will increase by increasing the duration of each of the tasks. In addition, the reason for the low level of the four tasks in their study can be attributed to the participants encountering unfamiliar concepts/topics (Sloan & Horton, 2019).

Efficiency: This is how quickly users are able to perform tasks after learning the pattern (Nielsen, 2012). In the test conducted according to the execution time of the determined tasks, only 20% of the participants were able to complete all the tasks on time. This suggests that the game needs to be improved in terms of efficiency. Hussain et al. (2014) scored an efficiency of 4 out of 5, although they did find some problems.

Game mechanics relate to the rule design and coding structure of a game (Demirbaş, 2020). In this study, participants often lost time waiting for the required numbers and failed due to time constraints. In this case, it was concluded that the game should be improved in terms of game mechanics in order to increase efficiency.

Memorability: Users should not have to re-learn how to use an application when they return to it after a while (Nielsen, 2012). If the interface design of different game scenes is consistent, users will quickly remember the system usage functionality (İşleyen et al., 2014). The levels in the current game (except for Task 1) were designed with similar features in terms of usage and game rules. Tasks 2 and 4 cover addition while Tasks 3 and 5 cover multiplication. This allows participants to play the game at all levels without having to learn new features. However, in light of the simple to complex principle, the tasks are similar but are designed in a structure that becomes increasingly difficult. This suggests that the memorability level of the game is low. However, this situation could be due to the increasingly difficult structure of the game. This can be improved by including more familiar design elements as these can increase memorability (Sloan & Horton, 2019).

Errors: This is about the number of mistakes that users make while using the application and their ability to correct them (Nielsen, 2012). The mistake usually made by participants when they started each level was that they forgot to pop at least two balloons. Providing feedback to users following mistakes can reduce the error rate. Similarly, in Zaki et al.’s study (2017), minimal errors were made in games, except an error of 50% in one game. Likewise, Hussain et al. (2014) observed the children's errors and stated a game was usable.

Satisfaction: This is the pleasure users experience while using the application (Nielsen, 2012). In the current study, once the game was over, participants were asked whether they liked the game or not. The positive responses received from all participants suggests they generally likes the game. Similarly, Hussain et al. (2014) recorded satisfaction at 4.2 and state that it was generally appreciated. One usability study recorded satisfaction at 97% (Saman et al., 2019) while another recorded it at 95% (Mahdi, 2017). Zaki et al. (2017) state that most users were satisfied with the games they looked at, but a few users were not satisfied with the games’ slow response. For users to enjoy playing a game and for the game to hold their interest, users must be able to complete the game and their achievements must be rewarded (AlDakhil et al., 2019). The current study concludes that in order to increase user satisfaction, the difficulty level of the game needs to be reduced and a reward system added.

Carrying out a usability test during the development of learning materials can reduce usability problems and increase the quality and efficiency of the system (Chang & Johnson, 2021). As a result of the usability test carried out for the current study, the efficiency of the game was evaluated and solutions were considered for the deficiencies identified.

The results of the study show that gender did not lead to a significant difference in the performance of learning tasks or on the duration of their execution. It can therefore be said that it is not important to use gender-specific designs in order to increase the effectiveness of serious games. Considering the learnability, efficiency, memorability, errors, and satisfaction criteria, it is thought that the learnability of the Unity-based game developed in the context of this study needs to be improved and amended in terms of efficiency, but the problems experienced regarding time are due to the fact that the users explored the game and were distracted by non-task objects during the tasks. When designing serious games, therefore, attention needs to be paid to the use of objects that will distract the user's attention from the learning content. In addition, the content of the task to be presented may need to be presented to students in a simple way. Memorability, bugs, and satisfaction are other aspects that need improvement in this game. In this context, it is important that the design used in the presentation of the content is as simple and plain as the teaching content of the game. A further consideration is that special attention should be paid to configuring the difficulty level of the game in order to increase user satisfaction. In addition, it has been observed that a well-structured reward system in serious games has positive contributions to the usability criteria of the game.

Although this study makes a valuable contribution to the literature and practice field of the development and usability of serious games developed for primary school mathematics courses, it does have some limitations. Our research only conducted a user-based usability study, the game was developed for one particular subject, math, and the study focused purely on second grade students. The study did not measure the effectiveness of the game with an experimental design. For future studies, we intend to develop our game in view of the deficiencies highlighted here and offer a more efficient and more usable learning material. It is also possible that the topics covered by the game can be increased in future studies and the game can be developed for different educational levels. Experimental studies could be conducted to measure the effectiveness of the game. Comparative studies could be conducted with large groups. The number of levels in the game can be increased according to the classes and longitudinal studies can be done. Usability studies can be extended using eye tracking. It is hoped that this study will contribute to studies aiming to develop digital educational games by suggesting ideas for reducing usability problems.