Keywords

1 Introduction

The games are part of almost all of our lives and by playing games, we practice certain physical, cognitive and social skills [1]. As the nature of the games has changed dramatically, the researchers are focusing on the potential of games. According to the literature, the potential of games is categorized into four domains: motivational, cognitive, social and emotional benefits [2]. These potential benefits of the games attracted many researchers in applying game mechanics in non-gaming context (commonly referred to as Gamification). The games designed for a primary objective other than pure entertainment are called serious games. The serious games are applied in the fields of education, scientific exploration, health care, defence, management, marketing, etc. The educational games are the subset of serious games. The main objective of the educational games is to help the player to construct knowledge about academic concepts and to train certain skills.

Learning is not limited to the classroom but every activity in life is an opportunity to learn. But there is a disconnect between education and life experiences. So, educational games necessitate the development of twenty-first-century skills and give methods for surveying these difficult to assess abilities. The twenty-first-century learning and innovative skill set are defined as critical thinking, creativity, collaboration and communication [3]. Skills relevant to the twenty-first century are different from skills the educational system currently values [4].

Using educational games, the learning can be made interesting and enjoyable. But many educational games are basic designs that are narrowly focused on academic content, target low-level literacy, provide drill and practice methods similar to worksheets and stress memorization of facts [5, 6]. There is no difference between educational games and worksheet, quiz. So, the educational game designs fail to engage students [7]. So, there is a necessity of a design framework for developing effective educational games.

2 Game Design Frameworks

The philosophy of the game design is the relatively underdeveloped field, at this time when so many games were developed and purchased [8]. Very few frameworks were proposed, among which the MDA framework [9] is the first framework, which helps to think game design in terms of three mutually dependent layers: mechanics, dynamics and aesthetics. The game is built out of basic components called mechanics. The material, rules, goals and control options are the basic components. The behaviour that results out of applying the game’s mechanics during the gameplay is dynamics. The changes in the mechanics affect the dynamics of the game. The aesthetics are the subjective experience, emotion and pleasure of the player. This MDA framework was widely used by the designers and also by the game design courses in academics.

Jesse Schell introduced the Elemental Tetrad [10], which comprises of four elements: mechanics, technology, aesthetics and story. Technology refers to the devices and frameworks used to actualize or convey the gameplay. Story refers to ‘the sequence of events that unfolds in your game’ [10]. Mechanics and aesthetics are the same as defined in the MDA framework. Later, Paul Ralph avoided the term story, instead, deconstructed story into three types of narratives: embedded, emergent and interpreted narratives, and proposed MTDA + N Conceptual Framework comprising of five elements: mechanics, dynamics, technology, aesthetics and narratives [11]. These are the evolution of the frameworks for regular game design.

2.1 Educational Game Design

The educational game design is different than the regular games. Developing games that are both fun and educational is a difficult challenge. In educational games, both fun and content has to be integrated so well, that the players get engaged with the game and enjoy the learning through games. There are some design frameworks found in the literature that claims these frameworks help to design educational games.

The experiential gaming model [12] presents the importance of experience in educational games. The main objective of experiential gaming model is to facilitate flow experience [13]. This framework can be used to design and study educational games. The design cycle [14] in this model, mainly focus on flow antecedents: clear goal, feedback, sense of control, focused attention, challenges, story, gamefulness and playability. The flow antecedents in this model were validated through a problem-solving game [14]. This framework provides the connection between constructive theory and gameplay but it claims to be utilized to design and analyse the educational games. It does not give the way to finish the educational game. In 2007, Adaptive Digital Game-based Learning Framework [15] was proposed. The author discussed the features of four frameworks and models for game-based learning in terms of two components: pedagogical and game design and proposed some features of the educational game. Three-Layered Thinking Model [16] was produced to incorporate game goals within the curriculum and game design. This model focus on four motivation factors: skill, concentration, challenge and pleasure. The usefulness of the model was described by designing and evaluating three educational games. This framework, for the most part, centres around the purpose of the game and its connection with the four motivation factors. In 2008, EFM: Model for Educational Game Design [17] was proposed as the framework describing the connection between motivation, flow experience and effective learning environment. The assessment tool to create effective learning environment by inspiring motivation through flow is absent. In 2009, an educational game design model was proposed that combines game design, pedagogy and learning content modelling [18]. The main focus of this model was on usability, multimodality and fun as game design elements, but it was not outlined in detail. The pedagogy and learning content modelling was combined with the emphasis on problem-solving, motivation, self-learning and syllabus matching. All the design components were mentioned but the relationship between them was not established in detail. The problem-solving was mentioned but the required assessments were absent. The Serious Game Design Assessment (SGDA) framework [19] also stress the purpose in every stage of the design and defines six main aspects for the design of a serious game that shall be successfully combined to achieve the game’s purpose: content, aesthetics/graphics, fiction/narrative, mechanics, framing and interaction. This framework helps to design educational game but also helps to examine the relationship between the design elements and the game purpose.

All the above-mentioned literatures focus on some aspect of educational game design and tried to address it through their frameworks. Table 24.1 shows the summary matrix of all the above-mentioned frameworks in terms of its background, design components, focus and its evaluation.

Table 24.1 Summary matrix of the existing educational game design frameworks
Full size table

In view of frameworks mentioned, the researchers are still exploring numerous dimensions in designing games. The perspective of the researchers in developing the educational framework is ranging from psychological and learning theories to game design theories. A variety of publications on game design and design strategies have been published in recent years [11, 19]. In most of these game design instructions, the serious game design is at best mentioned but not explained in detail. Some publications specifically focus on educational game design, but when it comes to questions regarding their assessment, significant tools are absent [19].

3 Evaluation Techniques

This section discusses the techniques that are used to evaluate educational games in terms of design and learning outcome. According to the literature, the researchers evaluated educational games by heuristic evaluation, user testing, cognitive walkthroughs, think aloud protocol, etc. The brief description of each technique is discussed in the following sections (Fig. 24.1).

Fig. 24.1
figure 1

Evaluation techniques

Full size image

3.1 Heuristic Evaluation

Heuristic evaluation [20] is originally used to evaluate user interfaces. In this method, a number of experts inspect the interface using a custom set of predefined criteria called heuristics. Heuristic evaluation performs better in detecting the serious problems with interfaces than other methods like usability testing and cognitive walkthrough; and is also cost-effective [21]. This technique has been embraced for the assessment of computer and board games, particularly in the initial phases of their design [22].

Later in 2010, Omar and Jaafar proposed Playability Heuristics Evaluation for Educational Computer Game (PHEG) [23]. The heuristics are divided by him into five aspects of educational games; interface, educational/pedagogical, content, multimedia, and playability. These heuristics are focused on an educational computer game, aiming to identify the issues in the digital game. Heuristic Evaluation for Educational Games (HEEG) is another heuristic for evaluating educational games [24]. This heuristic is based on existing heuristics such as HEP [22], PLAY [25], Game Flow and the Criteria for Designing Educational Computer Games. HEEG is applied on five games and evaluated by two researchers and a game developer.

One issue regarding heuristic evaluation is related to the heuristics themselves and their description [26]. The wide range and variety of the games that exist today make it challenging to design heuristics that are suitable for evaluating all kinds of games. Another issue with such a technique for evaluation is that a special class of people is required to perform such evaluation. Trained experts are not always that easy to come across. This sets a major drawback for performing such evaluation in mass quantities and frequent intervals.

3.2 User Testing

Most of the researchers evaluate educational games and the learning outcome using pretest—game—post-test, where the pre- and post-tests typically measure content knowledge [27]. In this method, participants were divided randomly into experimental group and control group. First, both the experimental group and the control group were pretested. Then, the experiment group is introduced to the game and they play for a certain period. Meanwhile, the control group takes part in regular traditional classes. After the designated period, both groups were tested. The pre- and post-tests are the questionnaires that have to be answered by the participants for measuring their content knowledge and usability of the game.

This technique can be used to test the game design and usability by the user’s review but it is not enough for testing adequate learning outcome. The questionnaire may also include questions regarding the topic that the game teaches. But the previous knowledge on the topic influence the answers given by the players.

3.3 Cognitive Walkthrough

In this method, the researchers usually use eye-tracking device and EEG, where the gameplay, eye movements and other emotions were recorded for further quantitative analysis. Watching the AVI files, annotating them and cutting them into smaller files help them to (a) concentrate on the most informative part of the video and (b) understand the player’s context. The extracted information includes the coordinates of the avatar, the coordinates of the objects that we want to track, and the coordinates of the cursor. The further analysis is based on these three categories of coordinate data. This technique is used to evaluate the digital games in terms of gameplay, graphics, attention, visual patterns, etc. [28, 29].

This technique is expensive and can be used only by experts who understand these devices. This technique is used to evaluate the game design and the learning outcome of the game by the experts.

3.4 Stealth Assessment Test

Stealth evaluation refers to Evidence-Centred Design-based assessments that are woven specifically and imperceptibly into the fabric of the gaming condition [30, 31]. In this assessment test, the learner performance data are continuously gathered during the course of play and inferences are made about the competencies. This assessment includes two key elements: evidence-centred design and feedback to support learning.

Evidence-Centred Design (ECD). Evidence-Centred Design is a conceptual framework that can be utilized to create assessment models, which in turn support the design of valid evaluations. The objective is to help assessment designers coherently align the cases that they need to make about students as well as the things that students say or do in relation to the unique situations and tasks of interest [32, 33]. The players learn through actions. There is a continuous interaction between player and game. So, the assessment of the learning cannot be isolated from the context. Therefore, the ECD is accepted to work well with digital games. The ECD structure encourages developers to connect what they need to evaluate and what students do in complex settings.

But there is a need for empirical evidence to use this kind of assessments in educational games. The competency model of ECD framework was developed only for problem-solving and systems thinking. There is a dearth of high-quality empirical evidence concerning how games in the classroom might impact the development of academics and twenty-first-century skills [34].

3.5 Comparison of Evaluation Techniques

The first two techniques mentioned in Table 24.2 are good enough for validating the game design but not enough for evaluating the learning outcome because game-based learning involves continuous interactions between learners and game. The cognitive walkthrough technique evaluates both the game design and learning outcome of the game by the experts but it is difficult to use by the educators who need immediate feedback. One primary challenge for instructors who want to employ games to support learning is assessment and immediate feedback. One approach to increase the quality and utility of assessment is to use evidence-centred design-based stealth assessment.

Table 24.2 Comparison of evaluation techniques used to evaluate educational games
Full size table

4 Conclusion

Design and evaluation are the important stages in the life cycle of any software development. The same rule can be applied to the game design. Many papers already mentioned that educational games research domain needs more empirical studies on its effectiveness towards education [3, 34, 35]. Along with the effectiveness of the games towards education, the other directions that are still lacking such as educational game design frameworks with detail assessment tools, techniques to evaluate the educational game design and learning outcome. The frameworks for designing educational games help designers to hold or support the theory behind the concept of educational game, which they want to achieve. The frameworks also assist the designers to establish the strong connection between the game goal and education goal. The existing frameworks focused only particular perspectives in designing educational games. There are multiple perspectives in designing games that have to be considered in educational game design such as learning theories, psychological aspects, assessment tools, etc.

Even though design frameworks assist designers in designing the educational games, the evaluation methods help in identifying areas for improvement and make designers realize educational game goals more efficiently. More adequate techniques to evaluate usability, learning outcome and effectiveness of the educational game are required. According to the present scenario, there is a disconnect between game design frameworks and evaluation techniques, probably this can be solved by incorporating the assessment tool in game design framework.