Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1 Introduction

Despite the growth of interest in serious games, there is still little systematic guidance concerning which kind of game is better for which purpose and how to assure a game fits the instruction required. The complexity of the field is clearly illustrated by, for instance, Connolly et al. [1] who in a recent review study classify games on genre, subject discipline and intended outcome. As a result, for developers the design and development of a game and for instructors the selection and application of a game can be quite an experimental process.

Design and development. Only recently, there exist a number of frameworks which attempt to integrate the knowledge and experience with regard to education, games and software [e.g. 2–4]. These frameworks are important tools to assist in the design of serious games. However, they are as yet not fully matured nor investigated as, indirectly, shown in the limited evidence on the effectiveness of serious games and the apparent difficulty in assessing the educational merits of serious games [1]. Moreover, these frameworks do not necessarily fit with the background of teachers.

Selection and Application. With the advent of social media it is widely accepted that teachers and learners are not only consumers but also may have an active role in sharing and co-creating content, debating and sharing opinions [5]. Social media such as social networks, online videos and wikis are not merely used to connect or entertain but also support informal learning [6] by enabling learners to ask questions, to debate and to share opinions and materials with other learners. Paradoxically, one strand of technology enhanced learning, i.e. game-based learning, aligns slightly with this development. Games, while there to experience, explore and collaborate, are almost exclusively designed by professionals, and because of their manifold appearances, difficult to value or comprehend. Generally teachers have insufficient knowledge about games and their beneficial usage in classrooms [7]. Educational games are considered fundamentally different from prevalent instructional paradigms [8]. Williamson [9] reports an urgent need for the training of teachers both at the initial training stage and the stages of continuous professional development, to pursue a better understanding of how to use games in their class-rooms as well as understanding the implications of games as cultural forms of young people’s lives. The general impression is that games require complex technologies and that games are difficult to organise and to embed in a curriculum [10]. The latter is of importance since the use of ICT and games, in particular, only tends to be successful if it closely fits with the existing teaching practice [11].

A way to support the game design and to support the application of a game would be to build upon a proven framework which integrates a sound instructional foundation, fits with teachers’ experiences and fits sufficiently with existing game principles. Huang and Johnson [12] propose using the 4C-ID model [13]. Key elements of this approach are authentic tasks, task classes, variation and increasing complexity in task classes, the distinction between supportive and procedural information and the proposed practice to automation of selected part-tasks. The overall design focus is on the integration and coordination of different levels of learning tasks and as such fits very well with existing game design practice. Recent studies [14, 15] confirmed the applicability of the model for game design and their embedding in education. Additionally, at a small scale 4C-ID was successfully applied by teachers for reviewing serious games and discussing how they could be used from their perspective in an educational setting [16, 17].

In the next section we will introduce the CHERMUG games and how 4C-ID influenced their design. We will conclude with a description of how the two games have been trialled following a set of questions derived from the 4C-ID model and summarise the participants’ findings.

2 CHERMUG Game Based Learning Design: Cognitive Task Analysis and 4C-ID

The CHERMUG project (www.chermug.eu) aimed to develop a digital game to support students in learning about research methods and statistics. Acquiring expertise in this area poses significant challenges for many students.

Our first step in developing a game was to identify the skills and competences required. A technique which has been developed to help analyse the higher level cognitive functioning required in tackling complex tasks is Cognitive Task Analysis (CTA). CTA is defined as “the extension of traditional task analysis techniques to yield information about the knowledge, thought processes and goal structures that underlie observable task performance” [18]. Cognitive task analyses have been used for a number of different purposes including the development of training and fit very well with the 4C-ID model since it can yield detailed information on the skills and competence required, how they decompose and are interconnected and stepwise should be learnt.

The CTA was based on a set of 13 interviews with experts selected for their knowledge of and involvement with teaching research methods and statistics [19] and resulted in a number of findings important for the design of the game. At the global level the research cycle (research question, data collection, data analysis, discussion & conclusion) presented in the briefing sheet was generally accepted by staff as providing a useful framework for presenting research methods to students. Moreover, designing a set of mini-games was seen as assisting the adoption of CHERMUG. At the detailed level, it was widely agreed that superficially observation suggests that statistics cause the main challenges, but in practice all steps in the research cycle are equally demanding. As one of the interviewees stated, opting to carry out a qualitative research methods project in order to avoid statistics does not necessarily pay off, since in practice analysing, interpreting and abstracting qualitative data can be very difficult. Moreover, the steps in the research methods cycle are tightly connected and choices made or lack of understanding at one step directly influences the following steps. Finally, the research question was perceived by many experts as providing a key challenge in developing a coherent approach to research methods.

The findings of the CTA were used in the design applying 4C-ID in the following way [20]. A set of mini-games was designed each based on an authentic and complete task (see Fig. 2 for an example of the research scenario of one of the games) distributed over three task classes, one with 3 games for qualitative research and two task classes on quantitative research (Table 1). Each of the task classes contained tasks with variation and increasing complexity. The supportive information is expected to be offered in advance in the class room or in an e-learning environment. Procedural information is implicit through the rigorous structure chosen for the games. Finally, a set of use cases [21] and best practices [22] was prepared to describe the possible use of the games within different settings. The examples chosen should be transparent for teachers and ease the use of the games in their curricula.

Table 1. The CHERMUG mini-games divided by their task classes and topics
Full size table

3 Workshop Program and Results

Workshop Setup. The workshop started with two presentations (1) an explanation of the 4C-ID model and (2) an introduction to the CHERMUG project. The participants were asked to briefly study the 4C-ID model (http://chermug-workshop.wikispaces.com/) and answer the following set of questions (Fig. 1) for the anticipated overall learning design in which the games would be used and for the games themselves, i.e.:

Fig. 1.
figure 1

The four component instructional design model [13].

Full size image
Fig. 2.
figure 2

The research scenario of one of the games.

Full size image
  1. 1.

    Learning tasks, i.e. the extent to which:

    • the game uses whole learning tasks;

    • the game uses a variety of tasks i.e. stressing different elements or examples;

    • the game uses task classes i.e. offering both simple and (more) complex examples;

  2. 2.

    Supportive information i.e. the extent to which:

    • the game uses supportive information;

  3. 3.

    Procedural information, i.e. the extent to which:

    • the game uses procedural information;

  4. 4.

    Part-task practice, i.e. the extent to which:

    • the game uses part-task-practice.

Next, the participants were asked to play a CHERMUG game (www.playgen.com/chermug) about research methods and statistics and Enercities (http://www.enercities.eu) a game about building a sustainable city each for around 15–20 min. They received the following assignment “Assume you are a teacher who wants to assess possible use of the game, i.e. look at the game and ‘guess’ which learning tasks can be supported/achieved by the game and/or additional materials”. Each of the games was followed by a questionnaire with questions that helped to classify serious game characteristics (based on the GaLA serious game evaluation form) and questions to assess if and how the game could be used in the given situation. Additionally, the questionnaire asked how the game was perceived in general and how useful the questionnaire was to help evaluating the applicability of a game and to determine any additional learning materials or activities needed. Finally, the workshop was closed with an open discussion of the experiences of the participants.

Participants and Results. Twelve persons participated in the workshop. All were experienced users, i.e. had average to good experience with using games in education. The participants worked in pairs (with exception of two) and completed one review round (the two CHERMUG games); the second game was played but not reviewed due to time limitations. The results were as follows.

First of all, the games were positively appreciated on various aspects including effectiveness, efficiency, usability and motivation. Moreover, it revealed that the participants agreed that the game design was based on whole learning tasks. The majority also agreed that the games made used of variability of practice from one to the next game (“different research methods are well introduced”). Questions how to use the games within the lesson practice with regard to variability and more tasks were clear but - in the discussion - the participants mentioned that they did miss the required teaching expertise in this domain to be able to come up with suggestions. With regard to procedural and supportive information the players were guided implicitly by the used scenario structure (“implicit by the scenario structure”), detailed explanations were only given if mistakes were made (“explanations are only given when something goes wrong”). The questions what additional procedural and supportive information should be added in the lessons were clear but again due to lack of domain knowledge difficult to answer (“yes, I can imagine there is a lot of additional (introductory) material”).

Finally, with regard to use of the 4C-ID model and the questions derived from it, most participants did positively appreciate the way the questions could help them to evaluate the applicability of games for their use. Critical remarks were focussed on the wordings of some questions which were sometimes ambiguous or required a better introduction to the model.

4 Conclusion

The key elements of the 4C-ID instructional design model i.e. authentic tasks, the distinction between supportive and procedural information and the proposed practice to automation of selected part-tasks fits well with game practice. The participants in the workshop could relatively easily reflect upon the games played with the help of the questions despite the challenge to play a game in a domain where they had limited familiarity. Equally important, the approach proposed helped to reflect beyond the game to the actual lesson plan in which games will be used and assisted in thinking about the design and the activities required to make best use of the games. The 4C-ID, though actually intended for curricula design, did give guidance on how to assess serious games and their usage.

Obviously given the size of the experiment (only two games, the limited time and number of participants) it is not possible to draw any firm conclusion as yet. Nevertheless, we do believe that the positive response to the model together with the research behind the model and the literature discussed, related to the use of the model for serious game design, warrant further research.