Keywords

1 Introduction

During the past decade the use of game-based learning solutions have increased. Nevertheless, it seems that the quality of game-based learning solutions varies a lot and the field is lacking of generally acknowledged theoretical frameworks for developing engaging and effective game-based learning solutions. Since digital games as a learning approach was proposed, game designers have faced challenges to integrate instructional and game design aspects [e.g. 1– 3]. For example, Quinn [2] has stated that educational games have to be well-designed to incorporate engagement that integrates with educational effectiveness. In fact, Habgood and Ainsworth [3] have proven that deep integration of game’s core mechanics and its learning content is crucial for creating intrinsically motivating and effective game-based learning solutions. However, the integration of content knowledge and game mechanics is not enough, but also the domain specific instructional knowledge should be considered when designing the core mechanics and dynamics of an educational game. Unfortunately, previous research has shown that systematic investigation of learning integration in games is lacking [4] and for example in the context of game-based rational number learning the intrinsic integration has been rare [5]. Thus, in order to teach educational GD for undergraduate students we designed a workshop that aimed to explore students’ game design preferences as well as to support the development of integrated educational GD mindset. With integrated mindset we refer to such educational GD practices in which instructional knowledge, content knowledge, game design knowledge as well as target group characteristics are considered in the game design decisions. In this paper we propose a framework for educational GD and game research that is built upon the idea of integrated educational GD mindset. The proposed framework helps to design learning activities in the educational GD subject and it was applied in the design of the game design workshop that was build upon an existing number line math game engine. Moreover, one aim of the workshop was to evaluate the feasibility of the idea to develop a game design workshop authoring tool around an existing game.

1.1 Related Work

According to Koivisto and Hamari [6] game-like characteristics of our world are increasing and gamification of activities, systems, and services has become more common. With gamification authors [6, pp. 191] refer to “designing information systems to afford similar experiences and motivations as games do, and consequently, attempting to affect user behavior”. Although most of the students have used gamified products and played digital games, only few are able to design and create games. According to Resnick et al. [7] digital fluency requires more than just interacting with media, it requires ability to collaboratively design, create, and invent with media. In this paper we report results of a study in which an emerging pedagogical strategy, learning by designing games was used to teach educational GD in a gamified workshop. The pedagogical idea behind learning by designing games approach relies on the assumption that game design activities help students to reformulate their understanding of the subject matter and express their personal ideas about both the subject and the designed game [8]. In line with this, Games’s [9] study in which game creation activities were investigated in an online Gamestar Mechanic environment (www.gamestarmechanic.com) showed that students can learn to analyze designs articulated by others as well as to articulate their own designs, which facilitates a deeper understanding of the expressive possibilities of games as a medium.

Kafai and Burke [10] have argued that we are currently witnessing a paradigmatic shift toward constructionist gaming in which students design games for learning instead of just consuming games created by professionals. They note that the popularity of Minecraft is the clearest indicator that constructionist gaming approach has arrived. According to recent literature reviews [10, 11], learning by designing/making games approach has been used to teach several subjects such as programming, computational concepts/strategies/thinking, mathematics, arts, and language & writing skills, but the studies in which the educational GD is a main learning objective are rare.

Nevertheless, a model of creative and playful learning [12], Smiley Model [13], and MDA (Mechanics, Dynamics, and Aesthetics) framework [14] can be applied in structuring learning by designing games activities. The creative and playful learning model distinguishes (1) orientation to tools, methods and the topic of learning, (2) game design and creation, (3) game play, and (4) elaboration, reflection and evaluation learning phases. The Smiley Model is a detailed framework for designing engaging learning experiences in games and it has been found to be useful in scaffolding the learning game design process [13]. On the other hand, the MDA framework proposes more general, formal approach, for game design and game research. MDA aims to bridge the gap between game design, game criticism, and technical game research. However, as the MDA is a general game framework we propose an extended CIMDELA (Content, Instruction, Mechanics, Dynamics, Engagement, Learning Analytics) framework to better fit to the educational game context (Fig. 1).

Fig. 1.
figure 1

CIMDELA framework for educational game design and game research

Full size image

In the proposed framework the aesthetics component is replaced with engagement component while it conceptualizes the meaning and consequences of aesthetics better in the educational game context. The mechanics generates dynamic system behavior, which aims to create certain experiences in players. The engagement component describes the desirable cognitive, behavioral, and emotional responses that interacting with the educational game system evokes in the player. The framework encourages to consider not only the designer’s perspective, but also player’s and teacher’s perspectives that in contrast to feature-driven design facilitates experience-driven and learning-driven design. The most important extension to the original MDA framework is addition of instructional knowledge, content knowledge, and Learning analytics components. With these components the CIMDELA framework aims to facilitate intrinsic integration of learning domain specific content knowledge, instructional knowledge, and learning analytics with mechanics, dynamics, and engagement components.

1.2 The Present Study

The educational GD workshop was held as a part of HYPE, which is a three-year long project funded by European Social Fund (ESF). The aim of HYPE is to develop new educational practices in the field of serious games in close collaboration with working-life partners and educational organizations. The project generates new patterns and models for cross-sectoral collaboration between various educational levels, industries, and organizations relying strongly on digital learning solutions. In this paper, we report a study in which we explored the usefulness of learning by designing games approach in teaching different viewpoints of educational GD. Furthermore, the aim was to evaluate students’ tendencies to consider both game design and instructional design aspects when making game design decisions as well as to evaluate the feasibility of the idea to develop a game design workshop authoring tool around the game used in the workshop.

2 Methods

2.1 Participants

Overall, 30 under-graduate students participated in the workshop. 20 of the participants were male and 10 females. The participants were 19–35 years old. The students attended the workshop as a part “Let’s make a game” -course organized by the HYPE-project. Due to the multidisciplinary nature of HYPE, the participants came from various backgrounds (e.g. technology, health-care, and cultural disciplines).

2.2 Workshop Design and Progress

The three-hour workshop consisted of three main phases. First, a short lecture about serious games and design of digital game-based learning solutions were given. The themes of the lecture were derived from the proposed CIMDELA framework and Smiley Model [13]. The aim of the lecture was to orient students to the game-based learning topic and to the tools used in the workshop. The lecture provided several design principles that students could apply in the second phase that consisted of game design and game play activities. Finally, in the third phase, debriefing was carried out and teams’ game design decisions were evaluated and elaborated.

The phases 2 and 3 formed a gamified part of the workshop. Several gamification elements such as points, leaderboard, tasks, teams, competition, narrative, and game rounds were used. Narrative elements were utilized to provide a context for the game design activities. The participants were divided into five design teams that worked in the same publishing company. The company had just started a new project in which they were developing a math game about fraction numbers for primary schools (10–12 years old pupils) and the task of the teams was to design a game demo for the next board meeting of the company. The teams competed between each other as only the best demo was promised to present in the forthcoming board meeting.

The teams did not have to start designing from the scratch, but the marketing department of the publishing company had already benchmarked the fraction games available in the market. Based on the benchmarking activities a prototype of the number line estimation game engine was already developed and several possible game mechanics were identified. Figure 2 shows the appearance of the game in the beginning of the workshop. The task of the teams was to decide what kind of game mechanics, aesthetics, and features were included in their final game demo and balance the dynamics. In practice the design activities were divided into eight game rounds that each included 1–3 design decisions to make. The rounds included decisions that were related to feedback and scaffolding mechanics, activable special skills, task types, scoring rules, obstacles, character movement, game balancing, and adaptation rules. Most of the decision tasks were designed in a way that the available options were conflicting with each other with respect to different design mindsets, entertainment mindset versus instructional mindset. For example, some mechanics would be nice additions to entertainment games enhancing emotional engagement, but in educational games these features could increase unnecessarily extraneous cognitive load and possibly even disturb learning or features could make the interpretation of learning analytics harder. The aim of such kind of conflicts was to trigger reflection between possible design choices and design perspectives and that way expand the design mindset of the participants.

Fig. 2.
figure 2

The appearance of the number line-based math game in the beginning of the workshop

Full size image

In practice the participants did not have to do any programming or graphical design, but only design decisions. In each round, the participants got design documents in which the possible design options were described (see Fig. 3 for an example of one design option). One round could include 1–3 design tasks. At the end of each round the teams provided their design decisions to the two game masters that implemented the decisions to the game through a game configuration file (implementation took couple minutes/team). After the implementation process the teams could play their game and experience the implemented features in action. During the game designing phase, the teams did not get any external feedback about their decisions, but from time to time they were reminded to pay attention also on instructional aspects and target group characteristics. Thus, the playing/testing experience was the only feedback channel during the design phase. Finally, in the debriefing phase all the decisions were elaborated and the achieved points from each decision was revealed. During the debriefing session a leaderboard was visible and players could see how their final rank was determined.

Fig. 3.
figure 3

An example of a design option of the scaffolding round

Full size image

2.3 Research Materials

The research material consists of observational data and participants game design decisions. In respect to ethical manner, the participants were aware of being observed [15]. As suggested in Merriam [16], the observation focused on physical environment, participants, activities, and interaction/communication and the observation themes were derived from the purpose of the study [16] focusing on decision making, game testing, game design justifications, and collaboration. The observation took place in the classroom and it followed the method of observing ongoing behavior within small groups. The observation was conducted by three researchers. One operated as a main observer and did not partake in any workshop activities. The observation sheet for the main observer was half-structured and it focused on the actual atmosphere, faced challenges, co-design activities, and concrete statements done by the workshop participants. Two other researchers (game masters of the workshop) observed the groups and their decision making in unstructured manner and discussed with participants during the workshop. The whole duration of the workshop (approx. 3 h) was observed.

3 Results

3.1 General Experiences About the Game Design Workshop

The learning by designing educational games concept functioned well and the participants seemed to like the designing activities. Most of the participants were fully concentrated on the topic, but a small number of students seemed to occasionally do some external activities while the rest of the team considered the design options. However, in most cases all the members of the teams took part in the final decisions of the design rounds. In general, the collaborative design approach worked well while participants had to justify their opinions and negotiate the final design decisions. Collaborative aspect clearly made the participants to consider design options from different perspectives. It was evident that 4/5 groups had a more or less clear leader, who strongly influenced on team’s decision-making process for example by making suggestions or making the team to vote on the options. One leader clearly indicated that he is a leader by stating that “I’m the leader, but I’m trying to be democratic so I’ll ask your opinions”.

Participants thought that 2-hour design session was appropriate. During the design phase the majority of the participants eagerly waited for the next round to start. In general, the round-based structure worked well and sequenced the activities into meaningful learning chunks. In fact, the most useful element of the design phase seemed to be the fast-based design-playing cycles. All the teams were eagerly waiting to see how their design choices changed the game as the statements of one team indicate “Yes, we will soon see what was the effect!… If that is critical adjustment for game play, I’m amazed”. The playing of the game after each round made participants to think and reflect on the design choices they had made as the following statements indicate: “So why it is now giving this?… Really? Oh, ok, it goes like THAT… Annoying, we didn’t think about that… Hey, this feedback was great. It really helps to teach what fraction is”. Usually there was one person in the group that played the game while others watched. Such approach facilitated discussions while the teams tested the game. On the other hand, the playing experience also helped participants to consider the options of the following design rounds in a meaningful context. There was also some frustration caused by the lack of competence during the workshop, both on the substance (fractions) and game design aspects as the following comments illustrate: “Argh, I don’t know how to count these!… Nooooo, help!… This is hard, even for a person as smart as I am.”

Based on the observational data, the competitive element did not play a significant role until the very end when the results were announced. This might be due to the fact that the competition was not emphasized in the beginning, but it was only a part of the background story of the workshop. Furthermore, the competition did not involve any external rewards or prizes and thus the extrinsic motivation was low. In fact, some of the participants even forgot that the competitive element was involved. For example, one participant was frustrated when he remembered the competition and realized that no prizes existed as the following statement shows “Yeah right, so you actually won just a good mood.” In spite of a lack of external rewards most of the players were very excited and the round by round evolving game seemed to provide an intrinsically motivating learning context. A real time scoring approach could have increased motivation and meaning of the competitive element. However, in several rounds there were overlapping learning themes and real time scoring would have undermined the instructional value of certain design rounds.

3.2 Students’ Game Designs and Design Trends

As a part of the debriefing phase the game designs of the teams were scored. The mean (sd) score was 100.67 (3.11). The best team scored 106 points from 120 maximum points. Overall, all the teams make quite good design choices and considered design decisions from several perspectives. However, in the individual level we identified students that had problems to take both instructional and game design aspects into account, but the team-based approach expanded their design mindset. The deeper analyses of teams’ design choices revealed several design trends.

The participants acknowledged the preconditions given in the beginning of the workshop and paid a lot of attention to target group characteristics. For example, one participant tried to figure out what is the best approach for kids and made a design suggestion of a new task type: “Text-based? I don’t think that would be the best fit for the children? Perhaps a visual approach?” Another participant reminded the rest of the team to remember the target group by saying that “We should carefully consider the target group and keep in mind what is the competence of a 3rd grader”. It became also evident that the participants were aware that math is not the most liked school subject and thus they did not want to design games that rely too much on mathematics that sometimes undermined the instructional value. For example, none of the teams did not include any mathematical obstacles in their game, but preferred non-mathematical obstacles that did not provide any instructional value. This was surprising while in many cases the teams tended to consider choices mostly from instructional point of view. Nevertheless, in several occasions the teams were able to consider the meaning of design choices from both learning and gameplay perspectives as the following argument shows “The last option would be the most accurate - most useful for learning, but the first one would be the funniest choice”. The teams clearly became aware of the fact that even if a chosen feature would be great for the gameplay experience, it might not enhance the actual learning and might even disturb learning. In this sense, some of the teams paid attention also to emotional aspects. For example, one of the teams discussed the meaning of a time limit and an obstacle type for playing experience: “If we add the time limit, it might cause more stress - which might hinder the actual learning… I would prefer choosing the ball, it causes no further stress and one would probably then learn more”.

As the teams wanted to create very positive learning experiences to the players, they preferred to use rewards and bonuses instead of such assessment mechanics that would more clearly reveal mathematical competences (learning analytics). In fact, students rarely thought the game from teacher’s perspective. The participants did not realize that the lack of competence metrics complicates the integration of game playing activities to classroom practices while teachers may face challenges to interpret reward-based in-game metrics. In fact, it became evident that the teams considered only the needs of the players that have difficulties in math. For example, all teams preferred adaptive scaffolding features over adaptive difficulty adjustment that would have taken also the needs of the high achieving players into account. Nevertheless, the design choices and discussions indicated that the participants understood the instructional power of scaffolding mechanics and elaborated feedback in contrast to simple corrective feedback.

If participants did not understand the mathematical meaning of a game mechanic they tended to justify their design decision with fun factors. For example, one team did not understand how the jumping movement might support understanding of unit fractions and they chose it, because they thought that it might be fun for the gameplay. We also noticed that sometimes teams decided to add a certain mechanic to their game, because they liked it so much and wanted to experience that mechanic in action. Thus, it would be reasonable to provide possibilities for participants to try all mechanics before making design decisions instead of just providing an example figures and descriptions of the mechanics as it was done in this study. All in all, the debriefing session was very useful, because the mechanics and design options were considered from different perspectives and participants could reflect on their own choices in the new light.

4 Discussion and Conclusions

This study contributes to the discussion of development of methods to teach design of game-based learning solutions. We explored the usefulness of learning by designing games approach to teach different viewpoints of educational GD to novice designers. We extended the MDA game framework [14] to CIMDELA framework (Content knowledge, Instructional knowledge, Mechanics, Dynamics, Engagement, Learning Analytics) and used the proposed framework to create such design tasks that require alignment of the game design and instructional design aspects. We observed students co-design activities and analyzed their game designs as a part of three-hour workshop that consisted of a short lecture, gamified design activities, and a debriefing session.

In general, the results showed that students were motivated and excited about the game design tasks that is consistent with results of Weitze [13]. However, although the design activities were gamified, the results indicated that the gamification was too loose and thus the gamified parts did not have large effect on students’ engagement and motivation. Gamification should be better integrated to core learning activities and students should be constantly aware of gamification elements and state of the system. On the other hand, the round-based structure worked well and sequenced the activities into meaningful learning chunks. The approach in which the students could test how their design decisions changed the gameplay motivated students a lot and the testing sessions facilitated deep discussions about the game designs. However, although the round-based structure worked well, the debriefing session in which the design options were finally elaborated at the end of the workshop were very important for learning and supported the development of the integrated design mindset.

Overall, the analyses of the observation notes indicated that learning by designing game-based learning solutions can be successfully utilized with novice game designers taken that the design activities do not require mastery of game programming. Our results provided evidence that some students had isolated design mindset and they had problems to consider design decision from multiple perspectives. There were students that tended to consider design choices either only from fun or instructional perspectives, but the team-based design activities helped them to expand their design mindset and consider decisions from multiple perspectives. The team-based design activities facilitated learning as students had to articulate and justify their design suggestions often leading to fruitful debate with co-designers. In our opinion the tasks in which design decisions can be justified from several perspectives (e.g. from entertainment or instructional design points of views) worked best as the participants had to consider the meaning of possible options more exhaustively. Although we did not measure learning outcomes, we believe that participants benefited from the design activities and their educational GD mindset expanded. To conclude, based on our experience the proposed CIMDELA framework can be a useful tool for designing learning activities about educational GD. However, we suggest to utilize also other frameworks such as Smiley Model [13] to form more detailed grounding of the design activities. In future, we aim to develop a game design workshop authoring tool around our math game engine to facilitate the organization of similar game design workshops.