Keywords

1 Introduction

Pedagogical Patterns (PPs) are a method of capturing and disseminating expert knowledge in teaching and learning, in the form of a reusable solution to a particular problem. They grow from the concept of design patterns and pattern languages more generally, which are common in computing disciplines. PPs offer a format and process for sharing successful teaching techniques, whilst allowing freedom of implementation to fit the needs of individual teachers [8, 12]. There has been little work addressing the potential advantages of utilising PPs in the analysis and design of serious games - either in terms of employing a PP mapping directly, or in discussion of methodologies for conducting such a mapping activity. Bridging this gap between pedagogy research and serious games design may lead to improved learning and gameplay outcomes from serious games, as their inherent interdisciplinarity is a challenge noted in recent literature [2, 32]. Therefore, a mutual understanding and interplay of concepts and language from serious game design processes (such as the Learning Mechanic - Game Mechanic (LM-GM) model [5, 22]) and PPs is a necessary step towards allowing serious game designers and researchers to take advantage of pedagogical patterns research. This would allow a more structured method to share and implement learning designed through serious and also afford another avenue for researchers to analyse the learning being created during serious game design.

In this work we present an analysis of a research project case study comprising twelve co-designed serious games from three distinct game jams focussed on games for cybersecurity. PPs have been applied to assist educators in computer science generally [7], but there is less evidence of this approach being undertaken outside of computing. No work has been identified to date on applying PPs to cybersecurity pedagogy. Gameplay and content of each co-designed game is analysed using the LM-GM framework and each game is then mapped onto pre-existing PPs, as defined by Bergin et al. [8]. The goal of this pattern matching is to investigate particular types of PPs that can emerge from serious game design, therefore serving as a foundation on which future researchers, designers, and game jam organisers can base their expectations and understanding of the pedagogy embedded within serious game designs. Furthermore, we present our process for this mapping activity and provide a mapping of concepts and language from the LM-GM model [5, 22] to the PPs format [8], thus providing common ground for communication between these two disciplines. Finally, we offer recommendations, based on our experiences, for further work that could improve collaboration processes and outcomes by the different disciplinary experts involved in serious game design.

2 Background

PPs can assist with teaching and learning technical subjects in particular, and their benefit is not restricted to a Higher Education context; they have also been applied in various training scenarios. PPs have a consistent structure comprising: Title; Context; Key Problem; Solution; and Further Information (including examples) [8]. Relationships with other PPs are included throughout these sections where relevant. Patterns are categorized in different groups: e.g. Active Learning, Feedback, Experiential Learning, Gaining Different Perspectives, and Teaching From Different Perspectives to aid educators in selecting and linking suitable techniques, and guidance is offered on how each pattern can be used at different timescales from minutes to an entire course [8]. Furthermore, Bergin et al. identify the opportunity for relating several patterns within a common problem space as a resource for solving complex problems, and actively solicit further contributions [8]. The literature notes that patterns are useful in scaffolding the solutions to multidisciplinary problems and enabling the swift articulation and transfer of expert knowledge to novices.

Patterns exist for many learning contexts, often needing to be specialised for particular disciplinary or industrial uses, for example, software design and security patterns [26]. For game making, game design patterns have been extensively studied since the early 00 s in an attempt to formally document gameplay [9], game mechanics, and non-player character (NPC) interactions [21]. Game design patterns are generally not presented as individual gameplay elements but provide designers with a set of interrelated connections, causal relationships and structures that can be used as building blocks towards the design of new games. Whilst not focused on pedagogy, the game design pattern approach is directly relevant to the PP approach proposed in this article. There is a need for development of PPs for specific topics and to move beyond PPs in classrooms to facilitate learning in a wider context, such as through professional practice.

Cybersecurity (the application domain of the serious game designs to be discussed in this paper) is an increasingly complex problem, in particular as a wider and more diverse group of people become creators and users of software, often without any training and operating outside the software industry. “Security is an abstract concept which combined with the dematerialized world of software systems is difficult to grasp, comprehend and experience” [13]. Cybersecurity combines technical, theoretical, professional, human, and social factors [23] and developing both skills and attitudes towards security is equally complex. Georgiou et al. [13] note the swift evolution of the subject and its related disciplines, a lack of technical, time, and expertise resources leading to poor coverage of cybersecurity in university-level programmes [11, 15], and the need for (and challenges of) acquiring crucial practical skills [25]. Perhaps due to some of these challenges, computer science and cybersecurity as disciplines have been quick to adopt developments in teaching and learning, such as gamification and game-based learning. Whilst gamification aims to increase motivation through rewards (for example, points, digital badges, and/or leaderboards), game-based learning relies on the combination of interaction mechanics related to pedagogy with those related to games to create Serious Game Mechanics, i.e. a “design decision that concretely realises the transition of a learning practice/goal into a mechanical element of gameplay” [5]. In this way ‘serious games’ (games which have an educational or training purpose) can not only model large complex systems but also increase a player’s understanding of, and confidence within, that system [10] whilst moving towards defined pedagogical goals.

Therefore, coding, software engineering in general, and cybersecurity specifically have been the topics of a wide range of recent game interventions from coding apps for young children [1], to gamified ‘Catch the Flag’ contests [29], to a fully-fledged game for developer-centred security [24], for example. However, despite evidence of rigour in the development of game-based learning for cybersecurity, there is little research linking game design decisions and Serious Game Mechanics (SGMs) with PPs to aid in their re-use for similar problems across the wider cybersecurity teaching and learning community. A number of papers consider design patterns or PPs in digital learning generally or in computing science topics such as programming (e.g. [18]) without specific reference to cybersecurity. In 2012, studies discussed the current state and value of design patterns in security and proposed a PP to translate remediation techniques into a pedagogically friendly format to address the adoption challenges faced by Malware incident responders [26, 27]. The authors also conclude that “in order for the pattern template and pattern knowledge repository to stay relevant, they should be maintained and updated by an open community of practitioners from the academia, security industry and security incident response practice” [27], however no more recent work on this topic has been identified. One recent study, focused on teaching computing science in general, noted the fruitful opportunity to map design decisions of a learning platform to PPs [31] but did not attempt this mapping and mentioned cybersecurity only briefly. Therefore, cybersecurity pedagogy and/or workplace training provides a useful and suitable context for this research.

In terms of game-based or gamified approaches, Hauge et al. note the potential for re-use of SGMs as design patterns driven by pedagogy [17] and there is evidence of studies which link PPs with sustainability in engineering [28] and professional training for accident prevention [6] but, until Georgiou et al. [13] no research linking PPs with a game-based approach in cybersecurity was identified. This paper defined ‘secure code’ mechanics based on attacks and mitigations around several cybersecurity themes. These were presented to workshop participants alongside selected gamification mechanics with the addition of learning elements and used to co-design game ideas for cybersecurity which mapped the secure code mechanics to suitable game elements. The authors note the centrality of pedagogical aspects to both cybersecurity as a discipline and in designing games to increase awareness and practice of secure coding [13] and present an abstracted design composition for cybersecurity games based on the co-designed game elements. This pattern was then analysed using the LM-GM framework for serious game design [5] and the resulting SGMs were presented, those that arose from the gamification toolkit used within the workshop and those that were generated by original participant contributions. This work provides a robust basis for building on by identifying SGMs created in a co-design process for cybersecurity games, mapping them to existing PPs, and reflecting on patterns that are potentially a good fit for this specific problem space. Therefore, this paper builds on the previous study’s findings and resources by using new data from serious game jam co-design events to: 1) broaden the identification and analysis of SGMs for cybersecurity beyond gamification and into game-based learning; 2) deepen the emphasis on pedagogical aspects of design; 3) map co-designed SGMs to PPs; and 4) produce recommendations for PPs for teaching and learning specific cybersecurity issues through game-based approaches.

3 Methodology

To improve multidisciplinary concept mapping and communication around the design of serious games, our methods also spanned disciplines. The relationships between pedagogical patterns, the LM-GM framework, a game jam method including Triadic Game Design [16], and our method of analysis are outlined below.

3.1 Serious ‘Slow’ Game Jam Method

The game jam format used to co-design the serious games discussed in this paper was the ‘Serious Slow Game Jam’ (SSGJ) [3]. For full details of the game setting and methodology, see this publication. This approach intended to ameliorate some of the key barriers to entry often found in traditional game jams, and added additional support and resources to reinforce and support the ‘serious’ aspect of the game jam. The SSGJ aimed to remove the main barrier of intense time pressure (the majority of traditional game jams take place over a 24–48 h period [20]), instead running over 5/6 working days spread over a 6–8 week period. Furthermore, participants are extensively mentored and supported in their serious game designs by:

  • The provision of a ‘provoking game’ [4] used to encourage reflection and discussion on cybersecurity and serious game design.

  • Support using the Triadic Game Design (TGD) methodology [16, 30].

  • Support in understanding and using gameplay loops (the various interaction cycles within a game) [14].

  • Domain expert mentors (in both cybersecurity and game design).

  • Three reference card decks to be used as design tools - one domain-specific (cybersecurity) and two covering Learning Mechanics and Game Mechanics (LM-GM).

These cards served to both provide scaffolding and inspiration for participants who may be new to any of the three areas (gaming, pedagogy and the domain), as well as providing a method of communication between expert mentors and participants. Outputs from this game jam approach include documentation such as: paper prototypes, gameplay loops, and game design documents. This documentation underpins the analysis described below.

3.2 Participants

Data was collected from three distinct Serious Slow Game Jams [3], Table 1 shows the format of these jams, the number of participants that took part and the number of game designs produced.

3.3 The Learning Mechanic-Game Mechanic (LM-GM) Model

The LM and GM cards were used during the game jams as design tools to create an LM-GM map, following the procedure defined in [5, 22]. The goal of the LM-GM mapping is to “highlight [a serious game’s] main pedagogical and entertainment features, and their interrelations” [5], Fig. 1 shows an example LM-GM mapping. This process not only clarified the learning and gaming aspects for participants but also functioned as clear documentation of design decisions, allowing mentors (and subsequent researchers) to understand the participants’ designs. This combination of a Learning and Gaming Mechanics results in a Serious Game Mechanic (SGM) which demonstrates how the pedagogical approach is implemented through gameplay. Resultant SGMs extracted from each game jam team’s designs were used to inform the process of PP mapping. LMs and GMs are defined in detail in [5] and for readers unfamiliar with this framework it is recommended that this associated literature is read in conjunction with our analysis and mapping, below.

Table 1. Summary of game jam participants.
Full size table
Fig. 1.
figure 1

An example of an LM-GM mapping created by a SSGJ team.

Full size image

3.4 Pedagogical Pattern Mapping Approach

Using the documentation of each game (including gameplay loops annotated with LM-GM cards, game design documents, and the LM-GM mapping exercise) we identified the LMs and GMs used in each game design and their various relationships which form SGMs. It should be noted that this data is based on the participants’ own use of the LM and GM toolkit and therefore may not be fully accurate (for example a relevant LM or GM that was not well understood by participants may have been omitted). However, teams were supported by both cybersecurity and serious game design mentors which increased the rigour of the process and gives confidence that the documentation produced is a good indication of the core SGMs for each game. Some SGMs were one-to-one relationships between a single LM and a single GM, however, in general these relationships were complex with one-to-many and many-to-many relationships. In order to analyse the respective popularity of different LMs and GMs across all co-designed games, they were then recorded as separate dyads (one-to-one mappings). Therefore, every LM-GM relationship was individually recorded, for example, a high-level LM such as Analysis was mapped separately to every GM with which it shares a relationship. The most commonly occurring SGM dyads (those that appeared in 25% (3) or more of the co-designed games) are shown in Fig. 5 in the Results section below, which also gives a sense of the simplicity or complexity of different SGMs. Note this figure is provided as an overview and is not intended to be used as an in-depth resource.

Note: to ease the use of the LM-GM framework in the SSGJ itself, we simplified some terminology from Arnab et al. [5]. LMs: Participation was removed as it is omnipresent in game-based learning; Explore and Discovery were combined as they are conceptually very similar; similarly, Imitation, Modelling, and Shadowing were combined with Observation; and Accountability and Responsibility were combined with Ownership. GMs: Communal Discovery was combined with Collaboration; Goods was combined with Tokens and separated from in-game Information; Pareto Optimal was retitled One-Player Must be Better-Off; Pavlovian Reactions was retitled Conditioning; Protégé Effects was retitled Learning by Teaching. Some definitions were rewritten in simpler terms, based on current pedagogy literature and clear examples of each concept in use were added to the toolkit cards. Therefore there may be minor differences in LM-GM terminology used here, nevertheless the concepts remain as defined. Example LM and GM cards can be seen in Figs. 2 and 3.

The next step was to establish relationships between the SGMs identified in the above process and the PPs defined in Bergin et al. [8]. A detailed reading of each PP was undertaken and each was mapped to the SGMs that are likely to implement the pattern within each game. The PPs are shown on the left and right sides of Fig. 5, linked to the relevant SGMs. See Bergin et al. for full definitions of each PP.

Fig. 2.
figure 2

Analysis LM Card.

Full size image
Fig. 3.
figure 3

Movement LM Card.

Full size image

4 Results and Discussion

4.1 LM-GM Framework Language

A particular advantage of the three game jams conducted as part of this work is that we were able to observe the LM-GM framework being used by a range of individuals, from primary school children, through to Master’s students. The most striking observation from this is the need for the LM-GM framework’s language to be revisited and revised for different user groups and desired learning outcomes. For example, participants in this case study were not familiar with the language of pedagogy and required support in their understanding of some LMs concepts. Conversely, if the same methods were used with educators, it is likely that the language of GMs would need additional explanation. One example of particular note is the usage of the Pavlovian Reactions Learning Mechanic. In the first two game jams, Pavlovian Response was an LM available to participants and even though expert mentors deemed that it was a mechanic present in some of the games developed, it wasn’t included in any of the participants’ chosen SGMs. However, when this mechanic was renamed to Conditioning for the final game jam that was aimed at schoolchildren, the mechanic was included in two separate designs. Whilst we are confident overall that most of the SGMs in the co-designed games were successfully documented, a lack of understanding of some of the LMs or GMs leading to their omission is a limitation of this research.

Fig. 4.
figure 4

Frequency of PPs which were mapped to participants’ SGMs.

Full size image

4.2 Frequency of Pedagogical Patterns Arising from the SGMs

Figure 4 shows the frequency of PPs which the SGMs were mapped to. Active Student, Repeat Yourself and War Game are considered to be inherent to serious games, hence these PPs have 100% frequency and are shown in their own section at the top of Fig. 5). Feedback is also present in 100% of the games, however this PP arises from specific SGMs identified by participants in their gameplay loop designs so is shown in the body of Fig. 5. Note that many PPs refer to specific formal educational contexts therefore all other PPs defined by Bergin et al. [8] that do not appear in Fig. 4 were not relevant in this mapping. It can be seen that PPs appearing in over 50% of the games are themed around incremental feedback based on manageable learning and/or actions, for example: Early Warning, Digestible Packets, Reflection, Spiral, Build and Maintain Confidence, Challenge Understanding, and Self-Test. Another theme emerging from these most popular PPs is that of applying lessons learned in the game to out-of-game contexts, for example: Expand the Known World, Multi Pronged Attack, One Concept Several Implementations, and Solution Before Abstraction. Figure 4 gives an indication of which PPs may be a fruitful starting point for game-based learning, both for cybersecurity and for topics beyond this case study.

4.3 SGM-to-PP Mapping

Figure 5 shows the proposed SGM-to-PPs mapping arising from our analysis. SGM groups are identified in the central column with interrelationships shown by linking arrows with an indication of how frequently the SGM dyad was used across the twelve co-designed games. All game designs employed the Feedback-Quick Feedback SGM (meaning the combination of the Feedback LM and Quick Feedback GM) and exactly half of the game designs used: Plan-Strategy/Planning, Analysis-Selecting/Collecting and Question & Answer - Question & Answer. For clarity, SGMs which were used in fewer than 25% of the games have been omitted from this diagram. Bounding boxes show where there is no overlap between SGM dyads - this gives an indication of those LMs and GMs which have many-to-many pairings and those which are more likely to be one-to-many or one-to-one.

The PPs are shown on the left and right of Fig. 5 as nodes with a white background. Linking arrows show relationships between each SGM group and the associated PP. Those shown on the left are more influenced by the linked LM(s) and those shown on the right are most influenced by the linked GM(s). Seventeen PPs were mapped to these particular SGMs (alongside the three aforementioned PPs inherent to all serious games). Again, the linking arrows show individual relationships and also give an indication of how wide-ranging a PP might be; most are linked to three or more different SGMs. However, it should be noted that this diagram shows only SGMs occurring in 25% or more of the games. Simply because SGMs and PPs occur more rarely does not mean that they lack value for specific approaches and learning outcomes. For example, 2 co-designed games demonstrated a very clear mapping from Identification and/or Ownership LMs to Roleplay GM, which in turn activates the Roleplay,Own Words, Build and Maintain Confidence, and Try it Yourself PPs. For players to identify with the wider problem of cybersecurity and take ownership over it was a major aim of the case study this paper is based on, and this particular SGM and related PPs would be considered very fruitful in, for example, workplace cybersecurity training. Recommendations for how to use the SGM-PP mapping diagram (Fig. 5) are detailed in Sect. 5, below.

Fig. 5.
figure 5

The Serious Game Mechanics to Pedagogical Patterns Mapping.

Full size image

Depth of Learning Design. Further reflections on Figure 5 identify a high use of SGMs that are aligned with lower-order levels of understanding (as defined in Bloom’s Extended Taxonomy [19]) such as Question & Answer, Repetition, and Feedback. This suggests that co-designers (based, as most of them were, in a didactic educational context) may associate serious games with simple quizzes, or have struggled to escape the model of demonstrating knowledge followed by feedback from an educator. This was particularly noticeable in the school-age teams. This shows that more support is needed in the game jam process, particularly during the ‘meaning’ phase of the Triadic Game Design process [16], in order to provide participants with the appropriate scaffolding to allow them to explore more higher-order learning designs. That said, 50% of games used much higher-order LMs such as Analysis and Plan and incorporated these incremental developments of understanding (or conceptually similar variations, such as the Cascading Information GM) into their core gameplay and feedback loops. Neither lower nor high level learning outcomes are ‘better’, rather it is important to note that the specific learning outcomes of a serious game will suggest suitable SGMs and PPs to use. Lower order SGMs are very appropriate for knowledge acquisition games whereas, as noted above, higher order SGMs leading to understanding and ownership of a problem space would likely be more suitable for behavioural change outcomes such as cybersecurity workplace training.

Salience of Participant-Chosen Serious Game Mechanics. Further to the above, during our analysis there was some evidence of participants selecting and documenting SGMs that were valid, but not necessarily the most salient for their purpose, as designed earlier in the game jam process. In other words, SGMs were correct but some were omitted that would most effectively and accurately convey the learning and gameplay intended. This reinforces the need to tailor language and support specifically to the participants, as mentioned above, and the limitation that a lack of understanding can affect documentation accuracy, even where the game itself is rigorously designed. For example, in the Serious Slow Game Jams conducted as part of this work the target audience was “code-citizens”, or people who are code-literate but may not have explicit software engineering training [13]. Therefore, the goal of these jams was not just knowledge acquisition but to encourage behavioural change to improve these code citizens’ software development habits and practices. The Identify/Generalisation - Metagame SGM, which could map to PPs: Wider Perspective, Expand the Known World, Solution Before Abstraction and Roleplay is one SGM and several PPs that we considered to be a particularly salient one for this particular context, however, this particular SGM was not recorded in any of the game designs produced by participants. On the other hand, Metagame was associated with Incentive and Ownership LMs in one game each, both by adult teams who were close to entering the world of work. This reflection shows that there can be a great deal of subjectivity and ‘fuzziness’ in the creation of some SGMs, so individual SGMs should not be taken as the only useful articulation of a particular serious game design approach. We also propose that this identifies the potential for reverse-engineering SGMs through PPs to either strictly ensure, or gently guide, designers towards learning and gameplay mechanics which are more salient in terms of the actual learning outcomes of the game being designed.

5 Usage of Pedagogical Pattern Mapping Diagram

We propose two approaches to using our PP mapping diagram (Fig. 5) in preparation for supporting serious game design depending on the disciplinary needs of the intended audience: 1) for educators and 2) for game designers. We reiterate that bridging the disciplinary gaps between all three aspects of a serious game (the subject, the pedagogy, and the gameplay) is widely noted in the literature as a priority. Therefore easing communication for experts in each of these three groups is crucial. For educators, familiar with pedagogical concepts but perhaps not game mechanics, we propose approaching the model via the PPs. For example, an educator may be involved in co-designing a serious game to teach students what photosynthesis is. The educator can use their expertise to identify an appropriate PP for their learning outcome (based on adapting the closest template from the PP source literature [8]). This could be an approach already being used to teach the topic that the educator wants to see reflected in a game-based approach, or they could be inspired by the patterns and elect to try a new, potentially more effective pattern. Once a pattern (or several) are selected, they are then able to see what SGM mechanics are related to, and most likely to deliver, that pattern in a serious game context. Conversely, beginning from SGMs and moving outward towards PPs may be the most fruitful approach to those coming from a game design background. By defining appropriate SGMs for a proposed serious game (or identifying them in an existing game), a game designer can then associate the approach with the wider pedagogical theory to validate (or question) a particular serious game design approach. This not only provides additional pedagogical resources to assist in game design, but can provide a shared language and improve communication between game designers and educators.

5.1 Example Scenarios of Each Approach for Cybersecurity

Rather than directly instructing her students of some appropriate ways to eliminate the vulnerabilities related to a particular cyber attack, an educator would prefer that the students figure out the answers for themselves, whilst simultaneously becoming more self-reliant. She identifies Test Tube as a fruitful PP. Using the SGM-PP mapping (Fig. 5) she identifies the Exploration/Discovery - Quick Feedback SGM. After reading the definition of both LM and GM she confirms her choice and shares this information with the game designer, who reads Test Tube in order to cement their understanding. They then suggest a way in which the game system can support experimentation with clear results to allow the players to draw their own accurate conclusions on the impacts of different cyber defence mechanisms. As they collaborate, both the educator and the designer use the mapping as a facilitation tool for knowledge exchange.

From a game designers perspective, the SGM-PP mapping (Fig. 5) could be used to adapt the learning design of an existing serious game from a different application domain. For example, a company approaches a serious games company to produce a game to support workplace learning and practice of secure coding. The company has previously designed a serious game about common nursing procedures that used the Incentive - Competition SGM (Fig. 5). Looking at the SGM-PP mapping they can see that this SGM is mapped to Build and Maintain Confidence, which seems appropriate. The company checks with their client who clarifies that this pattern should be emphasised but that Competition should be represented as their company against other companies, not employee against employee! The game designers then follow Build and Maintain Confidence back to other SGMs and note that Repetition - Cascading Information wasn’t in their nursing game, but would be valuable to add in the cybersecurity game, as it involves knowledge acquisition as well as developing professional practice.

6 Future Work

6.1 Dynamic Presentation of SGM-PP Mapping

Figure 5 is challenging to parse in its current, static format, therefore we propose designing and creating a dynamic, web-based presentation of the SGM-PP mapping, so that it can be navigated and read more easily. This would also allow rarer but still valuable SGM-PP mappings to be added without making the diagram unreadable.

6.2 Audience-Specific Language in the LM-GM Framework

The LM-GM framework is rooted in serious games theory and therefore can use terminology that may be inaccessible to game jam participants including (but not limited to) participants who are young, speak English as an additional language, or have limited experience of either games or pedagogy theory. This was our rationale for simplifying the framework language used in the SSGJ toolkit card decks. Further work towards making the LM-GM framework more accessible and understandable to different audiences is recommended.

6.3 Built-for-Purpose Subsets of the SGM-PP Mapping

As mentioned above, there are situations where lower-order learning is appropriate and others (such as the cybersecurity domain example used here) where imparting higher-order learning and behavioural change is the goal. Therefore, subsets of SGMs, PPs, or both could be curated by experts to facilitate co-designs with particular learning goals, desired learning, behaviours, or contextual limitations. For example, the goal of teaching young children to play piano using a mobile device would benefit from a subset focused on learning-by-doing and repetition which would not only be more usable and less overwhelming to co-designers but would also reduce the risk of inappropriate SGMs being included.

7 Conclusions

Pedagogical patterns focus on transferable yet adaptable solutions and disseminating expert knowledge within teaching and learning. With a similar goal focused on serious games design - in particular co-design between experts from different disciplines - we have presented an analysis and subsequent mapping of twelve co-designed serious game designs onto established pedagogical patterns. This results in the proposed SGM-PP mapping diagram for which we describe our methodology and provide examples of potential usage. We propose that mapping serious game mechanics to pedagogical patterns is a helpful resource and process as it facilitates knowledge exchange and accurate interdisciplinary communication between educators and game designers, an activity which is at the very core of effective serious game design.